Bird
Raised Fist0

How to Estimate QPS, DAU, Storage, and Bandwidth for Any System

HLDmediumInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/5
API Gateway (Nginx)

Receives user request to estimate QPS and DAU

API Gateway accepts HTTP REST request and validates input

Protocol:HTTP/REST
BeforeUser request not yet received by system
AfterRequest accepted and forwarded to Kafka topic
Kafka Cluster

Queues the request message for processing

Kafka partitions message by metric type for ordering

Protocol:Kafka
BeforeRequest message in API Gateway only
AfterRequest durably stored in Kafka topic partition
Processing Workers (Kafka Consumers)

Consumes request message and computes estimates

Workers read Kafka, calculate QPS/DAU, update Cassandra

Protocol:Kafka Consumer / CQL
BeforeRequest message pending in Kafka topic
AfterEstimates computed and stored in Cassandra Cluster
Cassandra Cluster

Stores computed metrics for durability

Writes metric data with replication for fault tolerance

Protocol:CQL
BeforeNo new metric data stored for this request
AfterMetric data durably stored and replicated in Cassandra Cluster
Read Serving Layer (Cassandra + Redis)

Serves cached or fresh metric data to clients

Reads Redis cache first; falls back to Cassandra if miss

Protocol:Redis GET / CQL
BeforeClient waiting for metric data response
AfterClient receives low-latency metric data response
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────────────────┐
│   API Gateway (Nginx)     │
└─────────────┬─────────────┘
              │ HTTP/REST
              ▼
┌───────────────────────────┐
│     Kafka Cluster         │
│  (Partitioned Topics)     │
└─────────────┬─────────────┘
              │ Kafka
              ▼
┌───────────────────────────┐
│ Processing Workers        │
│   (Kafka Consumers)       │
└─────────────┬─────────────┘
              │ CQL (Cassandra Query Language)
              ▼
┌───────────────────────────┐
│   Cassandra Cluster       │
│  (Distributed Storage)    │
└─────────────┬─────────────┘
              │ Redis GET / CQL
              ▼
┌───────────────────────────┐
│ Read Serving Layer        │
│ (Cassandra + Redis Cache) │
└───────────────────────────┘
Design Rationale

This architecture handles metric estimation requests by routing them through an API Gateway that accepts HTTP REST calls. The requests are queued in a Kafka Cluster, which decouples ingestion from processing and ensures ordered, durable message storage. Processing Workers consume Kafka messages to compute QPS, DAU, storage, and bandwidth estimates, then persist results in a Cassandra Cluster for durability and scalability. The Read Serving Layer combines Cassandra and Redis cache to provide low-latency reads, serving cached data when available and falling back to Cassandra otherwise. This design balances throughput, durability, and read performance for metric estimation workloads.

Primary Focus

✓

Practice

(1/5)
1. A globally distributed application requires strong consistency for writes but must tolerate network partitions gracefully. Which replication pattern best fits this scenario?
easy
A. Primary-replica replication with asynchronous replication to replicas
B. Multi-primary replication with asynchronous conflict resolution
C. Multi-primary replication with eventual consistency and last-write-wins
D. Primary-replica replication with synchronous writes to primary

Solution

  1. Step 1: Understand consistency requirements

    Strong consistency requires writes to be serialized and acknowledged before success.

  2. Step 2: Evaluate replication patterns

    Primary-replica with synchronous writes ensures all writes go through a single primary, preserving consistency. Multi-primary with async or eventual consistency cannot guarantee strong consistency.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Primary sync writes -> strong consistency, tolerates partitions by blocking writes [check]
Hint: Strong consistency -> single primary synchronous writes [check]
Common Mistakes:
2. In the SLO error budget tracking system, at step 4 the Error Budget Calculator receives aggregated error counts and total request counts. What state does it maintain immediately after processing these inputs?
easy
A. It resets the error budget to full capacity regardless of recent errors
B. It discards error counts if the total request count is below a threshold
C. It queues the error counts for asynchronous processing by the alerting system
D. It updates the current error budget remaining by subtracting recent error rate from total budget

Solution

  1. Step 1: Understand the role of the Error Budget Calculator at step 4

    It updates the remaining error budget by subtracting the recent error rate from the total allowed errors, reflecting current SLO compliance.

  2. Step 2: Exclude other states that contradict error budget tracking logic

    Resetting the budget ignores recent errors; queuing is done later by alerting; discarding counts would lose accuracy.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Error Budget Calculator updates remaining budget after aggregation [check]
Hint: Error budget updates subtract recent errors, not reset or discard [OK]
Common Mistakes:
3. A web application requires real-time bidirectional communication with low latency and minimal overhead for interactive gaming. Which technology best fits this requirement?
easy
A. Use WebSockets because they provide full-duplex communication with low latency suitable for interactive use.
B. Use Server-Sent Events because they support unidirectional streaming from server to client efficiently.
C. Use Long Polling because it is simpler to implement and works well for real-time bidirectional communication.
D. Use HTTP/2 multiplexing as it inherently supports bidirectional streaming without additional protocols.

Solution

  1. Step 1: Identify communication direction and latency needs

    Interactive gaming requires low latency and bidirectional communication, which Server-Sent Events and Long Polling do not efficiently provide.

  2. Step 2: Evaluate protocol capabilities

    WebSockets provide full-duplex communication with minimal overhead, ideal for real-time interactive applications. HTTP/2 multiplexing is not widely supported for bidirectional streaming in browsers and adds complexity.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    WebSockets enable low-latency bidirectional streams [check]
Hint: Bidirectional + low latency -> WebSockets [check]
Common Mistakes:
4. For a system requiring low-latency approximate frequency queries on a data stream with heavy skew, which approach is better and why?
medium
A. Use Count-Min Sketch with conservative update to reduce overestimation bias
B. Use a single large hash table with exact counts to avoid approximation errors
C. Use a Bloom filter combined with exact counters for heavy hitters
D. Use a HyperLogLog sketch to estimate frequencies with lower memory

Solution

  1. Step 1: Evaluate exact counts vs approximation

    Exact counts require large memory and high latency for heavy skew data streams.

  2. Step 2: Recognize Count-Min Sketch with conservative update

    Conservative update reduces overestimation bias, improving accuracy for skewed data.

  3. Step 3: Eliminate Bloom filter and HyperLogLog

    Bloom filters don't count frequencies; HyperLogLog estimates cardinality, not frequency.

  4. Final Answer:

    Option A -> Option A

  5. Quick Check:

    Conservative update Count-Min Sketch balances accuracy and memory for skewed streams [check]
Hint: Conservative update reduces bias in Count-Min Sketch for skewed data [check]
Common Mistakes:
5. At 10x traffic scale, which component in a reverse proxy architecture is most likely to fail first and why?
hard
A. Backend servers, because they cannot handle increased encrypted traffic
B. Reverse proxy, because SSL termination is CPU-intensive and stateful per connection
C. DNS servers, because they cannot resolve increased domain queries fast enough
D. Load balancers, because they cannot distribute traffic evenly at high scale

Solution

  1. Step 1: Identify bottleneck in reverse proxy at scale

    SSL termination requires CPU and maintains state per connection, making reverse proxy the first to saturate under 10x traffic.

  2. Step 2: Eliminate other options

    Backend servers handle decrypted traffic; DNS queries scale better with caching; load balancers scale horizontally more easily.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Reverse proxy SSL termination CPU and state bottleneck at scale [check]
Hint: SSL termination CPU load breaks reverse proxy first [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard