Bird
Raised Fist0

Back-of-the-Envelope Estimation with Real Numbers - Servers, Storage, Bandwidth

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
🏗️
Back-of-the-Envelope Estimation with Real Numbers - Servers, Storage, Bandwidth
Master precise capacity planning with concrete numbers for large-scale systems
mediumSystem Design Interview ApproachFreq: common
⚙️
Functional Requirements
  • Support ingestion of 2 billion daily active users generating messages
  • Store messages with retention of 30 days for compliance and analytics
  • Provide read access with 100ms P99 latency for user queries
  • Support peak write throughput during high traffic hours
  • Ensure data durability with replication across data centers
  • Expose APIs for message write, read, and metadata queries
📊
Non-Functional Requirements
  • Handle 2 billion DAU generating 40 messages per user per day
  • Achieve 99.99% availability SLA for write and read operations
  • Maintain 100ms P99 latency on reads under peak load
  • Data retention for 30 days with replication factor of 3
  • Support peak write QPS up to 1.5 million messages per second
❓
Clarifying Questions to Ask
  • What is the average and peak message size in bytes?
  • What is the expected read-to-write ratio for messages?
  • Is the message data immutable or can it be updated/deleted?
  • What is the acceptable data loss tolerance during failures?
  • Are there any constraints on data center locations or network bandwidth?
  • What is the expected query pattern: user-centric or global search?
Out of Scope
✗ End-to-end encryption and security protocols
✗ Detailed UI/UX design for client applications
✗ Complex analytics or machine learning on message content
✗ Multi-region geo-distribution beyond replication
📌
Assumptions
  • 2 billion DAU
  • 40 messages per user per day
  • 100 bytes average message size
  • Retention period of 30 days
  • Replication factor of 3
⚡
QPS Calculation
write qps2,000,000,000 × 40 / 86,400 ≈ 925,926 msg/sec
peak write qpsAssuming 1.5× peak factor: 925,926 × 1.5 ≈ 1,388,889 msg/sec
read qpsAssuming 2 reads per message on average: 925,926 × 2 ≈ 1,851,852 msg/sec
💾
Storage Estimate
daily storage gb(2B × 40 × 100 bytes) / (1024³) ≈ 7.45 TB/day
retention storage gb7.45 TB × 30 days ≈ 223.5 TB
with replication gb223.5 TB × 3 ≈ 670.5 TB
🖥️
Server Estimate
write servers[object Object]
storage nodes[object Object]
🏗️
Architecture Diagram
Architecture Diagram
          +-------------+
          |  Clients    |
          +------+------+
                 | Write/Read Requests
          +------+------+
          | API Gateway|
          +------+------+
                 |
          +------+------+
          | Kafka Cluster|
          +------+------+
                 |
          +------+------+
          | Cassandra DB|
          +------+------+
                 |
          +------+------+
          | Cache Layer |
          +------+------+
                 |
          +------+------+
          | Monitoring |
          +-------------+
🧩
Components
API Gateway
Handles client requests, routes writes to Kafka and reads to cache/DB
Why thisNginx with Lua for high concurrency and flexible routing with low latency
Why notAWS API Gateway has higher latency and cost at 1.5M QPS scale
Kafka Cluster
Durable, scalable write buffer for incoming messages with partitioning
Why thisKafka supports 1M+ writes/sec per cluster with partitioned logs and replication
Why notRabbitMQ lacks horizontal scalability and throughput at this scale
Cassandra DB
Distributed storage for messages with high write throughput and linear scalability
Why thisCassandra's write-optimized LSM trees and tunable consistency fit 1.3M writes/sec
Why notPostgreSQL requires manual sharding and does not scale linearly at 1M+ writes/sec
Cache Layer (Redis Cluster)
Serve low-latency reads for hot data and reduce DB load
Why thisRedis Cluster supports sharding and sub-millisecond reads with TTL for stale data
Why notMemcached lacks persistence and advanced data structures needed for metadata
Monitoring & Alerting
Track system health, latency, throughput, and trigger alerts
Why thisPrometheus + Grafana provide scalable metrics collection and real-time dashboards
Why notCloudWatch has higher latency and cost at this scale
🔄
Data Flow
1
Clients → API Gateway
Send message write/read requests
API Gateway authenticates and routes requests
2
API Gateway → Kafka Cluster
Publish incoming messages to Kafka topic partitions
Kafka partitions messages by user ID for load balancing
3
Kafka Cluster → Cassandra DB
Consume messages from Kafka and write to Cassandra
Asynchronous consumer ensures durability and eventual consistency
4
Clients → API Gateway
Read requests routed to Cache Layer
Cache hit returns data immediately, else fallback to Cassandra
5
Cache Layer → Cassandra DB
Fetch data on cache miss
Cache updated asynchronously after DB read
6
Monitoring → All Components
Collect metrics and logs
Enables SLA tracking and anomaly detection

Practice

(1/5)
1. A company needs to store large amounts of unstructured data like images and videos with metadata accessible via HTTP APIs. Which storage type best fits this use case?
easy
A. Object storage, because it stores data as objects with metadata and supports HTTP access.
B. Block storage, because it provides low-latency random access to raw data blocks.
C. File storage, because it offers hierarchical directories and POSIX file system semantics.
D. In-memory storage, because it offers the fastest access to large files.

Solution

  1. Step 1: Identify storage needs for unstructured data with metadata

    Object storage stores data as objects with customizable metadata and supports HTTP APIs, ideal for images/videos.
  2. Step 2: Eliminate other storage types

    Block storage is low-level and lacks metadata support; file storage is optimized for hierarchical file systems but less scalable for massive unstructured data; in-memory storage is volatile and unsuitable for large persistent files.
  3. Final Answer:

    Option A -> Option A
  4. Quick Check:

    Object storage = scalable unstructured data + metadata + HTTP [check]
Hint: Object storage = objects + metadata + HTTP access [check]
Common Mistakes:
2. In the message delivery trace, at Step 4 the Delivery Worker checks Bob's presence TTL in the presence service cache. What happens if Bob's presence TTL has expired at this step?
easy
A. Delivery Worker immediately marks message as delivered without retry
B. Delivery Worker retries presence check indefinitely until Bob reconnects
C. Delivery Worker forwards message to Bob's last known server regardless of presence
D. Delivery Worker queues message for offline storage and does not attempt real-time delivery

Solution

  1. Step 1: Trace Delivery Worker decision at Step 4 -- Bob's presence TTL

    If Bob's presence TTL expired, the system treats Bob as offline but still attempts delivery to his last known server to avoid message loss.
  2. Step 2: Eliminate other options

    Immediate delivery marking is incorrect as message may not be delivered; queuing for offline storage happens later; indefinite retries cause blocking; forwarding to last known server is the fallback.
  3. Final Answer:

    Option C -> Option C
  4. Quick Check:

    Delivery Worker forwards to last known server if presence expired [check]
Hint: Expired presence TTL -> fallback to last known server delivery [check]
Common Mistakes:
3. For timeline generation at scale, which architecture better handles users with millions of followers and why?
medium
A. Pure pull-based: generate timelines on-demand by querying follow graph and tweets
B. Push-based fan-out: write each tweet to all followers' timelines synchronously
C. Hybrid approach: push fan-out for users with few followers, pull on read for celebrities
D. Store timelines in a global sorted set updated asynchronously for all users

Solution

  1. Step 1: Analyze push vs pull trade-offs

    Push fan-out is efficient for low-follower users but expensive for celebrities; pull is costly on read latency.
  2. Step 2: Evaluate hybrid approach

    Hybrid uses push for most users, pull for celebrities, balancing latency and write amplification.
  3. Step 3: Eliminate other options

    Pure push (A) causes huge write amplification; pure pull (C) causes high read latency; global sorted set (D) is hard to scale and update asynchronously.
  4. Final Answer:

    Option C -> Option C
  5. Quick Check:

    Hybrid push-pull balances write cost and read latency [check]
Hint: Hybrid push-pull fits follower count extremes [check]
Common Mistakes:
4. At 10x scale, which component in the ML data infrastructure is most likely to become the bottleneck first, and why?
hard
A. The feature store's online serving layer, because in-memory caches cannot scale linearly with data volume
B. The training pipeline's batch processing cluster, because compute resources scale easily with more nodes
C. The feature metadata registry, because relational databases handle metadata efficiently at scale
D. The ingestion Kafka cluster, because partition count limits throughput and rebalancing causes downtime

Solution

  1. Step 1: Analyze scaling challenges per component

    Feature store caches can be sharded; batch clusters scale horizontally; metadata registries are smaller and less frequently accessed.
  2. Step 2: Identify Kafka bottleneck

    Kafka throughput depends on partition count, which is hard to increase dynamically. Rebalancing partitions at scale causes downtime and latency spikes, making Kafka ingestion the first bottleneck.
  3. Final Answer:

    Option D -> Option D
  4. Quick Check:

    Kafka partition limits and rebalancing cause ingestion bottlenecks [check]
Hint: Kafka partitions limit throughput; rebalancing causes downtime [check]
Common Mistakes:
5. At 10x scale in user requests and live traffic updates, which component is most likely to fail first and why?
hard
A. The live traffic ingestion pipeline, as message queues saturate and delay update propagation
B. The Road Graph database, due to inability to handle increased write volume for live updates
C. The Routing Engine servers, because shortest path computations become CPU-bound and latency spikes
D. The client-side map rendering, due to increased data volume overwhelming browser memory

Solution

  1. Step 1: Analyze scaling bottlenecks

    Road Graph DB (A) can be sharded; ingestion pipeline (C) can scale horizontally; client rendering (D) is local and less critical.
  2. Step 2: Identify Routing Engine CPU bottleneck

    Routing Engine performs complex shortest path computations per request; 10x load causes CPU saturation and latency spikes, failing first.
  3. Final Answer:

    Option C -> Option C
  4. Quick Check:

    Routing Engine CPU-bound under 10x load causes first failure [check]
Hint: Routing Engine CPU saturates first under 10x load [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard