Bird
Raised Fist0

Performance - P50 P95 P99 Latency and Why Averages Lie

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/5
API Servers

Receive latency metric request

API Servers accept client request for P50, P95, P99 latency metrics

Protocol:WebSocket
BeforeNo latency request received yet
AfterLatency request queued for processing, request ID assigned
Kafka Cluster

Buffer latency events

API Servers publish raw latency events to Kafka topic partitioned by service

Protocol:Kafka
BeforeLatency events not yet queued
AfterLatency events durably stored in Kafka for ordered processing
Apache Flink Stream Processor

Compute real-time latency percentiles

Flink consumes Kafka events, updates P50, P95, P99 aggregates using windowed state

Protocol:Kafka Consumer / Internal State
BeforePercentile aggregates outdated
AfterPercentile aggregates updated with latest latency data
Cassandra Database

Persist raw and aggregated data

Flink writes raw latency events and updated aggregates to Cassandra

Protocol:CQL
BeforeCassandra missing latest latency data
AfterCassandra contains up-to-date raw and aggregate latency records
Dashboards and Alerting

Query and visualize latency metrics

Dashboards query Cassandra for P50, P95, P99 and display results; alerts trigger if thresholds breached

Protocol:CQL
BeforeDashboards show stale or no latency data
AfterDashboards display accurate percentile latencies; alerts reflect current state
🏗️
Architecture Diagram
Architecture Diagram
┌─────────────────────┐
│     API Servers     │
│  (REST/gRPC Front)  │
└──────────┬──────────┘
           │ WebSocket
           ▼
┌─────────────────────┐
│   Kafka Cluster     │
│ (Event Queue for    │
│  latency data)      │
└──────────┬──────────┘
           │ Kafka
           ▼
┌───────────────────────────────┐
│ Apache Flink Stream Processor  │
│ (Real-time percentile calc)   │
└───────────────┬───────────────┘
                │ CQL
                ▼
┌─────────────────────┐
│ Cassandra Database  │
│ (Stores raw latency │
│  and aggregates)    │
└──────────┬──────────┘
           │ CQL
           ▼
┌───────────────────────────┐
│ Dashboards and Alerting   │
│ (Visualize P50, P95, P99  │
│  and trigger alerts)      │
└───────────────────────────┘
Design Rationale

This architecture captures latency metrics with a focus on percentiles (P50, P95, P99) rather than averages, which can be misleading due to outliers. API Servers receive requests and publish raw latency events to Kafka, ensuring ordered, durable buffering. Apache Flink processes these streams in real-time, computing percentile aggregates using windowed stateful operators. Flink persists both raw and aggregated data to Cassandra, a scalable wide-column store optimized for time-series data. Dashboards query Cassandra to visualize latency percentiles and trigger alerts on SLA breaches. This design highlights why averages can hide tail latency spikes, while percentiles provide a clearer performance picture.

Primary Focus

✓

Practice

(1/5)
1. A distributed messaging system processes millions of events per second. Some messages fail processing due to transient errors and must be retried without blocking the main queue. Which component best handles isolating these failed messages for later inspection and retry?
easy
A. A Dead Letter Queue (DLQ) configured to capture failed messages separately
B. A cache layer like Redis to temporarily store failed messages for retry
C. A high-throughput relational database with retry logic in application code
D. A load balancer that redirects failed messages to a backup server

Solution

  1. Step 1: Identify the role of Dead Letter Queues

    DLQs isolate failed messages from the main queue, enabling retries without blocking normal processing.

  2. Step 2: Evaluate alternatives

    Relational DBs (A) add latency and complexity; Redis (C) lacks durability guarantees; load balancers (D) do not handle message state or retries.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    DLQ isolates failures for retry without blocking main queue [check]
Hint: Dead Letter Queues isolate failures, not caches or DBs [OK]
Common Mistakes:
2. In the ticket booking request flow, at Step 4 the Seat Locking Service checks if the seat is currently locked. What happens if the seat lock has expired just before this check?
easy
A. The service treats the seat as available and allows the booking to proceed
B. The service returns a locked status, preventing booking until timeout resets
C. The service queues the booking request until the seat lock is refreshed
D. The service throws an error indicating seat lock state is inconsistent

Solution

  1. Step 1: Trace Seat Locking Service behavior at Step 4

    The service queries the distributed cache for the seat lock key with TTL.

  2. Step 2: Understand TTL expiration effect

    If the TTL expired, the lock key is removed, so the service sees no lock and treats the seat as free.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Expired TTL means no lock key -> seat available [check]
Hint: Expired TTL means lock gone -> seat free to book [OK]
Common Mistakes:
3. You must design a distributed cache for a social media feed that tolerates network partitions and prioritizes availability over strict ordering. Which consistency model and architecture combination best fits this scenario?
medium
A. Eventual consistency with asynchronous replication and conflict resolution
B. Strong consistency with synchronous replication to all nodes
C. Causal consistency with vector clocks and dependency tracking
D. Read-your-writes consistency with client session affinity

Solution

  1. Step 1: Analyze availability and partition tolerance needs

    Eventual consistency with asynchronous replication maximizes availability during partitions, tolerating temporary inconsistencies.

  2. Step 2: Compare with other models

    Strong consistency blocks during partitions; causal consistency adds overhead and complexity not needed here; read-your-writes focuses on client guarantees, not global availability.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Eventual consistency favors availability and partition tolerance [check]
Hint: Eventual consistency maximizes availability during partitions [check]
Common Mistakes:
4. For a system requiring guaranteed message processing with minimal latency, which retry strategy is best suited when transient failures occur?
medium
A. Manual retries triggered by operators after failure alerts
B. Immediate retry with exponential backoff and jitter
C. Scheduled retries using a delayed queue with fixed intervals
D. Infinite immediate retries until success or manual intervention

Solution

  1. Step 1: Analyze retry strategies for latency and reliability

    Scheduled retries with delayed queues (B) balance retry timing and system load, reducing hot loops.

  2. Step 2: Evaluate other options

    Immediate retries (A) risk overload; manual retries (C) add latency; infinite retries (D) cause resource exhaustion.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Delayed queue retries balance latency and failure handling [check]
Hint: Delayed retries avoid overload, unlike immediate or manual retries [OK]
Common Mistakes:
5. At 10x traffic scale, which API Gateway component is most likely to become the bottleneck first, and why?
hard
A. Rate limiting store, because centralized counters cause contention and latency spikes
B. Authentication service, because token validation is CPU-intensive and stateful
C. API Gateway routing logic, because routing tables grow exponentially with services
D. Backend microservices, because they cannot horizontally scale as easily as the gateway

Solution

  1. Step 1: Identify scaling challenges

    Rate limiting often relies on centralized counters or stores (e.g., Redis) that become contention points under high request rates.

  2. Step 2: Eliminate other options

    Backend microservices, because they cannot horizontally scale as easily as the gateway is incorrect as backend services typically scale horizontally. API Gateway routing logic, because routing tables grow exponentially with services is unlikely as routing tables are cached and optimized. Authentication service, because token validation is CPU-intensive and stateful is less likely as token validation is stateless and can be cached.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Centralized rate limiting store bottlenecks first at scale [check]
Hint: Centralized counters bottleneck rate limiting at scale [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard