Distributed tracing in HLD - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Purpose of Distributed Tracing in Microservices

Which of the following best describes the primary purpose of distributed tracing in a microservices architecture?

A To automatically scale microservices based on request volume.

B To monitor and visualize the flow of requests across multiple services to identify latency and errors.

C To encrypt all data transmitted between microservices for security.

D To store all logs from microservices in a centralized database.

💡 Hint

Think about how distributed tracing helps understand request paths and performance.

Distributed tracing tracks requests as they travel through different services, helping identify where delays or errors occur.

❓ Architecture

intermediate

Key Components of a Distributed Tracing System

Which set of components is essential for implementing a distributed tracing system?

A Load balancers, API gateways, and service registries.

B Database replicas, cache servers, and message queues.

C Trace context propagators, trace collectors, and a trace storage backend.

D Authentication servers, firewalls, and VPN gateways.

💡 Hint

Focus on components that handle trace data collection and storage.

Distributed tracing requires propagating trace context, collecting trace data, and storing it for analysis.

❓ scaling

advanced

Scaling Trace Data Collection in High-Traffic Systems

In a high-traffic distributed system, what is the best approach to scale trace data collection without overwhelming storage and processing?

A Send trace data synchronously with each request to ensure completeness.

B Store all trace data indefinitely without filtering.

C Disable tracing during peak traffic to reduce load.

D Implement sampling to collect traces only for a subset of requests.

💡 Hint

Consider how to reduce data volume while keeping useful information.

Sampling reduces the amount of trace data collected, balancing detail and resource use.

❓ tradeoff

advanced

Tradeoffs Between Synchronous and Asynchronous Trace Reporting

What is a key tradeoff when choosing between synchronous and asynchronous trace reporting in distributed tracing?

A Synchronous reporting ensures trace completeness but increases request latency; asynchronous reduces latency but may lose trace data on failure.

B Synchronous reporting reduces request latency but may lose trace data; asynchronous increases latency but guarantees trace completeness.

C Both synchronous and asynchronous reporting have the same impact on latency and data loss.

D Synchronous reporting is only used for error traces; asynchronous is used for all other traces.

💡 Hint

Think about how reporting method affects user experience and data reliability.

Synchronous reporting waits for trace data to be sent, adding latency but ensuring data is captured. Asynchronous sends data in background, reducing latency but risking data loss if failures occur.

❓ estimation

expert

Estimating Storage Requirements for Distributed Tracing Data

A system handles 1 million requests per minute. Each trace generates approximately 10 KB of data. If the system stores traces for 7 days, what is the approximate storage needed for trace data?

A Approximately 100 TB

B Approximately 70 TB

C Approximately 1 TB

D Approximately 10 TB

💡 Hint

Calculate total data per minute, then multiply by minutes in 7 days.

1 million requests/min × 10 KB = 10 GB/min. 7 days = 7 × 24 × 60 = 10080 minutes. Total = 10 GB × 10080 = 100800 GB ≈ 100 TB.

Practice

1. What is the main purpose of distributed tracing in a system?
TraceID and spans are used to:

easy

A. Balance load between servers automatically

B. Store user data securely in a database

C. Track requests as they move through multiple services

D. Encrypt communication between clients and servers

Solution

1. Step 1: Understand distributed tracing basics

   Distributed tracing helps track how a request flows through different parts of a system.

2. Step 2: Identify the role of TraceID and spans

   TraceID links all parts of a request, and spans represent each step or service involved.

3. Final Answer:

   Track requests as they move through multiple services -> Option C

4. Quick Check:

   Distributed tracing = Track requests [OK]

Hint: TraceID connects steps in request flow [OK]

Common Mistakes:

• Confusing tracing with data storage
• Mixing tracing with load balancing
• Thinking tracing encrypts data

2. Which of the following is the correct way to represent a span in distributed tracing data?

easy

A. "traceId:abc123;spanId:xyz789;operation:db_query"

B. { "traceId": "abc123", "spanId": "xyz789", "operation": "db_query" }

C. [ "traceId", "spanId", "operation" ]

D. abc123xyz789

Solution

1. Step 1: Identify correct data format for spans

   Spans are usually represented as structured data like JSON objects with keys and values.

2. Step 2: Compare options

   { "traceId": "abc123", "spanId": "xyz789", "operation": "db_query" } shows a JSON object with clear keys for traceId, spanId, and operation, which is standard.

3. Final Answer:

   { "traceId": "abc123", "spanId": "xyz789", "operation": "db_query" } -> Option B

4. Quick Check:

   Span = JSON object with keys [OK]

Hint: Spans use JSON key-value pairs [OK]

Common Mistakes:

• Using arrays instead of objects
• Using plain strings without structure
• Using XML tags incorrectly

3. Given the following trace data snippet:

{"traceId": "t1", "spans": [{"spanId": "s1", "parentId": null, "operation": "frontend"}, {"spanId": "s2", "parentId": "s1", "operation": "backend"}, {"spanId": "s3", "parentId": "s2", "operation": "db_query"}]}

What is the correct order of operations for this request?

medium

A. frontend -> backend -> db_query

B. db_query -> backend -> frontend

C. backend -> frontend -> db_query

D. frontend -> db_query -> backend

Solution

1. Step 1: Analyze parent-child relationships

   The span with parentId null is the root: frontend (s1). Backend (s2) is child of frontend, and db_query (s3) is child of backend.

2. Step 2: Determine operation order

   Request flows from frontend to backend to db_query following parent-child links.

3. Final Answer:

   frontend -> backend -> db_query -> Option A

4. Quick Check:

   Parent-child order = frontend to db_query [OK]

Hint: Follow parentId links from root span [OK]

Common Mistakes:

• Ignoring parentId null as root
• Reversing operation order
• Mixing span order in array with execution order

4. You notice that some spans in your distributed tracing system have missing traceId values. What is the most likely cause?

medium

A. The tracing library failed to propagate the traceId across services

B. The database storing traces is full

C. The network is too slow to send trace data

D. The user did not authenticate properly

Solution

1. Step 1: Understand traceId propagation

   TraceId must be passed along all services handling the request to link spans correctly.

2. Step 2: Identify cause of missing traceId

   If traceId is missing, it usually means the tracing library or instrumentation did not forward it properly.

3. Final Answer:

   The tracing library failed to propagate the traceId across services -> Option A

4. Quick Check:

   Missing traceId = propagation failure [OK]

Hint: Missing traceId means propagation error [OK]

Common Mistakes:

• Blaming storage or network issues
• Confusing authentication with tracing
• Assuming traceId is optional

5. You want to design a distributed tracing system for a microservices architecture with hundreds of services. Which approach best ensures scalability and minimal overhead?

hard

A. Store all trace data locally on each service without aggregation

B. Trace every request fully and store all spans in a single centralized database

C. Send trace data synchronously during each request to avoid data loss

D. Use sampling to trace only a subset of requests and store spans in a scalable backend

Solution

1. Step 1: Consider scalability challenges

   Tracing every request fully creates huge data and overhead, which is not scalable for hundreds of services.

2. Step 2: Evaluate best practices

   Sampling reduces data volume by tracing only some requests, and scalable backends handle storage efficiently.

3. Step 3: Assess other options

   Synchronous sending blocks requests, and local storage prevents global analysis.

4. Final Answer:

   Use sampling to trace only a subset of requests and store spans in a scalable backend -> Option D

5. Quick Check:

   Sampling + scalable storage = scalable tracing [OK]

Hint: Sample requests to reduce tracing load [OK]

Common Mistakes:

• Tracing all requests causing overload
• Synchronous sending blocking services
• Not aggregating trace data centrally