Bird
Raised Fist0

Performance - P50 P95 P99 Latency and Why Averages Lie

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Performance - P50 P95 P99 Latency and Why Averages Lie
Designing systems with latency percentiles to reveal true user experience beyond averages
mediumNon-Functional Requirements FrameworkFreq: common
⚙️
Functional Requirements
  • Collect latency measurements for each request in real-time from distributed services
  • Compute latency percentiles (P50, P95, P99) over configurable time windows (e.g., 1 min, 5 min)
  • Provide APIs and dashboards to query latency percentiles per service, endpoint, and region
  • Support alerting when latency percentiles exceed defined thresholds
  • Store raw latency data for at least 7 days for historical analysis
  • Aggregate latency metrics with tags such as service version, region, and request type
📊
Non-Functional Requirements
  • Support 2 billion daily active users generating 40 requests per user per day
  • Achieve 99.99% availability for latency metric queries
  • Latency percentile computation must complete within 30 seconds after data ingestion
  • System must handle peak write QPS of approximately 925K writes/sec
  • Storage must retain 7 days of raw latency data with replication for durability
❓
Clarifying Questions to Ask
  • What is the expected request volume and data size per latency measurement?
  • Are latency measurements sampled or collected for every request?
  • What is the acceptable staleness for latency percentile data (real-time vs batch)?
  • Should percentiles be computed globally or segmented by dimensions like region or service?
  • What retention period is required for raw latency data and aggregated metrics?
  • Are there existing monitoring or telemetry systems to integrate with?
Out of Scope
✗ Designing the application services that generate latency data
✗ Detailed UI/UX design of dashboards and alerting systems
✗ Security and authentication mechanisms for the monitoring system
📌
Assumptions
  • 2 billion DAU generating 40 requests per user per day
  • Each latency measurement record is approximately 100 bytes
  • Retention period for raw data is 7 days
  • Replication factor of 3 for durability
⚡
QPS Calculation
write qps2B × 40 / 86400 ≈ 925,926 writes/sec
peak write qpsAssuming 2× peak factor: 925,926 × 2 ≈ 1,851,852 writes/sec
read qpsAssuming 10% of write QPS for queries: 925,926 × 0.1 ≈ 92,593 reads/sec
💾
Storage Estimate
daily storage2B × 40 × 100 bytes = 8,000 TB/day (8 PB/day)
7 day storage8 PB × 7 = 56 PB raw data
with replication56 PB × 3 = 168 PB total storage
🖥️
Server Estimate
ingestion serversKafka cluster with 1000 brokers to handle 1.85M writes/sec (each broker ~2K writes/sec capacity)
storage nodesCassandra cluster with 2000 nodes (each node ~100 GB/day write capacity) to store 56 PB raw data with replication
🏗️
Architecture Diagram
Architecture Diagram
          +----------------+
          |  Client Apps   |
          +-------+--------+
                  |
                  v
          +-------+--------+
          |  Instrumented  |
          |  Services      |
          +-------+--------+
                  |
                  v
          +-------+--------+
          |  Kafka Cluster |
          | (Ingestion Q)  |
          +-------+--------+
                  |
                  v
          +-------+--------+
          |  Stream Proc   |
          | (Apache Flink) |
          +-------+--------+
                  |
                  v
          +-------+--------+
          |  Cassandra DB  |
          | (Raw + Agg)   |
          +-------+--------+
                  |
                  v
          +-------+--------+
          |  API Servers  |
          +-------+--------+
                  |
                  v
          +-------+--------+
          |  Dashboards   |
          +----------------+
🧩
Components
Kafka Cluster
Ingests raw latency measurement events at high throughput
Why thisKafka supports partitioned, durable, and scalable ingestion with >1M writes/sec per cluster
Why notRabbitMQ lacks horizontal scalability and throughput for 1.85M writes/sec peak
Apache Flink Stream Processor
Computes latency percentiles (P50, P95, P99) in near real-time over sliding windows
Why thisFlink supports stateful stream processing with low latency and windowed aggregations
Why notApache Spark Streaming has higher latency and batch micro-batch model unsuitable for 30s SLA
Cassandra Database
Stores raw latency data and aggregated percentile metrics with high write throughput and replication
Why thisCassandra scales linearly for writes, supports time-series data with TTL, and multi-datacenter replication
Why notPostgreSQL requires manual sharding and cannot handle 925K writes/sec without bottlenecks
API Servers
Serve latency percentile queries and provide data to dashboards and alerting systems
Why thisStateless REST API servers behind load balancers scale horizontally to handle 100K+ QPS reads
Why notGraphQL adds complexity and latency overhead for simple percentile queries
Dashboards and Alerting
Visualize latency percentiles and trigger alerts on SLA violations
Why thisGrafana integrates with Cassandra and supports real-time dashboards and alerting rules
Why notCustom UI requires significant engineering effort and lacks mature alerting features
🔄
Data Flow
1
Instrumented Services → Kafka Cluster
Publish latency measurement events per request
Events include timestamp, service, endpoint, latency, and tags
2
Kafka Cluster → Apache Flink
Consume events for real-time percentile computation
Flink processes events in sliding windows (e.g., 1 min) to compute P50, P95, P99
3
Apache Flink → Cassandra DB
Write aggregated percentile metrics and raw events
Aggregates stored with TTL; raw data stored for 7 days
4
API Servers → Cassandra DB
Query percentile metrics and raw latency data
Supports queries by service, endpoint, region, and time range
5
API Servers → Dashboards and Alerting
Serve data for visualization and trigger alerts
Alerts configured on percentile thresholds (e.g., P99 > 100ms)

Practice

(1/5)
1. For storing large blobs with frequent read access and occasional updates, which storage architecture is best suited to optimize cost and performance in this blob store design?
medium
A. Store entire blobs in a distributed key-value store with synchronous replication
B. Use a relational database with BLOB columns and transactional updates
C. Use chunked blob storage with immutable chunks and metadata pointers for versioning
D. Store blobs as single large files on a network file system with locking for updates

Solution

  1. Step 1: Analyze chunked storage with immutable chunks

    This approach enables efficient caching, deduplication, and cheap versioning by reusing unchanged chunks.

  2. Step 2: Eliminate other options

    Key-value stores struggle with large blobs and synchronous replication adds latency; network file systems have locking bottlenecks; relational DBs are inefficient for large binary data.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Chunked immutable storage optimizes cost and versioning [check]
Hint: Immutable chunking enables cheap versioning and caching [check]
Common Mistakes:
2. If the Kafka cluster used for buffering document update events becomes unavailable, what is the immediate failure mode in the full-text search indexing pipeline?
medium
A. The system automatically switches to a backup message queue with identical ordering guarantees
B. Document update ingestion blocks, causing the entire indexing pipeline to stall and new updates to be lost if not retried
C. The indexing workers continue processing stale events from local cache, causing outdated search results
D. The inverted index becomes corrupted due to missing update events

Solution

  1. Step 1: Understand Kafka unavailability impact

    Without Kafka, ingestion of new document updates cannot proceed, causing blocking or stalling.

  2. Step 2: Evaluate options

    Indexing workers do not have local caches of events (A) and cannot continue processing. Automatic failover to a backup queue with identical ordering (C) is a designed mitigation. Index corruption (D) does not happen immediately; missing events cause stale results but not corruption.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Failover to backup queue prevents pipeline stall [check]
Hint: Kafka outage stalls ingestion unless backup queue with ordering [check]
Common Mistakes:
3. For a system requiring cost optimization with infrequent data access but occasional retrieval within hours, which storage architecture is preferable?
medium
A. Implement warm storage with HDDs and cold storage with object storage, migrating data based on access patterns
B. Use only hot storage with SSDs for all data to ensure fastest access
C. Archive all data immediately to tape storage to minimize cost
D. Store all data in cold storage with no tiering to simplify design

Solution

  1. Step 1: Analyze access and cost requirements

    Data is infrequently accessed but needs retrieval within hours, so cold storage alone is too slow.

  2. Step 2: Evaluate tiering benefits

    Warm storage offers moderate cost and latency; cold storage reduces cost further. Tiering balances cost and retrieval time.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Warm + cold tiering balances cost and retrieval latency [check]
Hint: Warm + cold tiering balances cost and latency [check]
Common Mistakes:
4. If Zookeeper's session timeout mechanism fails to detect a crashed leader node promptly, what is the most likely failure observed in the leader election process?
medium
A. Multiple nodes simultaneously assume leadership causing split-brain
B. Leader election proceeds normally due to backup leader election mechanism
C. Followers wait indefinitely for leader node removal, blocking leader election
D. Clients receive stale configuration but leader election continues

Solution

  1. Step 1: Understand session timeout role

    Session timeout detects leader failure by ephemeral node removal.

  2. Step 2: Failure of timeout detection

    If timeout fails, ephemeral node remains, so followers do not start election; no split-brain occurs.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Backup election mechanisms prevent split-brain despite timeout failure [check]
Hint: Session timeout failure -> ephemeral node stays -> no immediate election [check]
Common Mistakes:
5. At 10x traffic scale during a flash sale, which component is most likely to fail first and why?
hard
A. The relational database because it cannot handle 10x write throughput without sharding
B. The API Gateway because it maintains stateful user sessions and cannot horizontally scale easily
C. The cache layer because TTL expiration floods the database with cache misses
D. The inventory service because distributed locking causes contention and latency spikes

Solution

  1. Step 1: Identify stateful components under load

    API Gateway maintains user session state (e.g., WebSocket connections), which are sticky and stateful, limiting horizontal scaling.

  2. Step 2: Compare other components

    Relational DB (A) can be sharded; cache (C) TTL floods are mitigated by pre-warming; inventory locking (D) is optimized with partitioning. API Gateway statefulness causes connection limits and bottlenecks first.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    API Gateway stateful sessions limit horizontal scaling [check]
Hint: Stateful API Gateway limits scaling under 10x load [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard