Correlation IDs in Microservices - Scalability & System Analysis

Scalability Analysis - Correlation IDs

Growth Table: Correlation IDs in Microservices

Users/Requests	What Changes?
100 requests/sec	Correlation IDs added to trace requests across services; simple logging; minimal overhead.
10,000 requests/sec	Logs grow large; need centralized logging and tracing system; correlation IDs help link logs.
1,000,000 requests/sec	Massive log volume; tracing data stored in distributed tracing systems; correlation IDs critical for performance analysis and debugging.
100,000,000 requests/sec	Tracing data must be sampled; correlation IDs used with high-performance telemetry; storage and processing optimized for scale.

First Bottleneck

The first bottleneck is the logging and tracing infrastructure. As requests grow, the volume of logs and trace data linked by correlation IDs overwhelms storage and processing systems.

Scaling Solutions

• Centralized Logging: Use systems like ELK stack or Splunk to aggregate logs with correlation IDs.
• Distributed Tracing: Implement tracing tools (e.g., Jaeger, Zipkin) that use correlation IDs to track requests end-to-end.
• Sampling: Sample traces to reduce data volume while keeping useful insights.
• Asynchronous Logging: Use non-blocking loggers to avoid slowing services.
• Compression and Archival: Compress logs and archive old data to save storage.
• Load Balancing: Distribute tracing and logging workloads across multiple servers.

Back-of-Envelope Cost Analysis

Assuming 1 million requests/sec, each generating 1 KB of trace/log data with correlation IDs:

• Data generated per second: ~1 GB/sec
• Data per day: ~86 TB
• Storage needed: High-capacity distributed storage with compression
• Network bandwidth: Must support ~8 Gbps+ for log shipping
• Processing: Distributed tracing systems must handle millions of trace spans/sec

Interview Tip

When discussing correlation IDs scalability, start by explaining their role in tracing requests across services. Then identify logging/tracing infrastructure as the bottleneck. Propose solutions like centralized logging, distributed tracing, and sampling. Quantify data volume and discuss trade-offs between detail and performance.

Self Check

Your database handles 1000 QPS. Traffic grows 10x. What do you do first?

Answer: Since correlation IDs mainly affect logging/tracing, the first action is to ensure the logging and tracing infrastructure can handle 10,000 QPS. Implement sampling or increase logging system capacity before scaling the database.

Key Result

Correlation IDs help trace requests but create large logging and tracing data. The first bottleneck is logging infrastructure, which must scale with centralized systems, sampling, and distributed tracing.

Practice

1. What is the primary purpose of a Correlation ID in microservices?

easy

A. To balance load between servers

B. To encrypt data between services

C. To track a single request across multiple services for easier debugging

D. To store user session information

Solution

1. Step 1: Understand the role of Correlation ID

   A Correlation ID is a unique identifier attached to a request that travels through multiple services.

2. Step 2: Identify its main use

   This ID helps developers trace and debug the flow of that request across distributed systems.

3. Final Answer:

   To track a single request across multiple services for easier debugging -> Option C

4. Quick Check:

   Correlation ID = request tracking [OK]

Hint: Correlation ID links logs of one request across services [OK]

Common Mistakes:

• Confusing Correlation ID with encryption keys
• Thinking it balances load
• Assuming it stores user data

2. Which of the following is the correct way to pass a Correlation ID between microservices?

easy

A. Add it as a custom HTTP header in the request

B. Include it as a query parameter in the URL

C. Store it in a database before each call

D. Embed it inside the response body

Solution

1. Step 1: Review common practices for passing metadata

   Metadata like Correlation IDs are typically passed in HTTP headers to keep requests clean and consistent.

2. Step 2: Evaluate options

   Query parameters can be altered or logged insecurely; storing in DB is inefficient; response body is too late for tracking.

3. Final Answer:

   Add it as a custom HTTP header in the request -> Option A

4. Quick Check:

   Correlation ID in headers = best practice [OK]

Hint: Use HTTP headers to pass Correlation IDs [OK]

Common Mistakes:

• Using query parameters which can be insecure
• Storing IDs in database for each call
• Embedding IDs in response body instead of request

3. Consider this simplified code snippet in a microservice receiving an HTTP request:

def handle_request(request):
    correlation_id = request.headers.get('X-Correlation-ID')
    log(f"Start processing request {correlation_id}")
    # ... process ...
    log(f"End processing request {correlation_id}")

What will be logged if the incoming request has header X-Correlation-ID: abc123?

medium

A. No logs will be generated

B. Start processing request abc123
End processing request abc123

C. Start processing request X-Correlation-ID
End processing request X-Correlation-ID

D. Start processing request None
End processing request None

Solution

1. Step 1: Extract Correlation ID from headers

   The code uses request.headers.get('X-Correlation-ID') which returns the header value if present.

2. Step 2: Check the header value in the request

   The request has X-Correlation-ID: abc123, so correlation_id will be 'abc123'.

3. Final Answer:

   Start processing request abc123 End processing request abc123 -> Option B

4. Quick Check:

   Header value read correctly = logs with abc123 [OK]

Hint: Headers.get returns value or None; here value exists [OK]

Common Mistakes:

• Assuming header key is logged instead of value
• Thinking None is logged when header exists
• Believing no logs are generated

4. A developer notices that Correlation IDs are missing in logs of downstream services. Which is the most likely cause?

medium

A. The Correlation ID header is not forwarded in outgoing requests

B. The Correlation ID is too long and gets truncated

C. The logging system does not support string messages

D. The services are using different programming languages

Solution

1. Step 1: Understand Correlation ID propagation

   Correlation IDs must be passed along with every outgoing request to maintain traceability.

2. Step 2: Identify common propagation mistake

   If downstream logs miss the ID, it usually means the header was not forwarded properly.

3. Final Answer:

   The Correlation ID header is not forwarded in outgoing requests -> Option A

4. Quick Check:

   Missing ID in logs = header not forwarded [OK]

Hint: Always forward Correlation ID header downstream [OK]

Common Mistakes:

• Blaming length truncation without evidence
• Assuming logging system limitation
• Thinking language differences cause missing IDs

5. You design a microservices system where each service generates its own Correlation ID if none is provided. What is a potential problem with this approach?

hard

A. It ensures better security by hiding original IDs

B. It improves performance by reducing header size

C. It simplifies logging by having unique IDs per service

D. It breaks the traceability because multiple IDs exist for the same request

Solution

1. Step 1: Analyze the effect of generating new IDs per service

   If each service creates a new Correlation ID, the original request's trace is lost.

2. Step 2: Understand impact on traceability

   Multiple different IDs for one request make it impossible to follow the full request path across services.

3. Final Answer:

   It breaks the traceability because multiple IDs exist for the same request -> Option D

4. Quick Check:

   Multiple IDs = broken traceability [OK]

Hint: One Correlation ID per request across services only [OK]

Common Mistakes:

• Thinking multiple IDs improve security
• Assuming performance improves with new IDs
• Believing unique IDs per service simplify logs