Bird
Raised Fist0

Why System Design Exists and What Single Machines Cannot Do

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Why System Design Exists and What Single Machines Cannot Do
Understanding the fundamental limits of single machines and the need for distributed system design
mediumSystem Design Interview ApproachFreq: common
Prerequisites:System Design FundamentalsCAP Theorem
⚙️
Functional Requirements
  • Support horizontal scaling of data storage and processing beyond single machine limits
  • Provide fault tolerance and high availability despite hardware failures
  • Ensure data consistency and partition tolerance under network partitions
  • Enable low-latency read and write operations at large scale
  • Support concurrent access by billions of users
  • Allow incremental scaling without downtime
📊
Non-Functional Requirements
  • Handle 2 billion daily active users (DAU)
  • Sustain 925,000 writes per second at peak
  • Achieve 99.99% availability SLA
  • Maintain 100ms P99 latency for reads and writes
  • Retain data reliably for at least 30 days
❓
Clarifying Questions to Ask
  • What is the expected peak QPS and data volume?
  • What are the consistency requirements (strong, eventual)?
  • Is the system read-heavy, write-heavy, or balanced?
  • What are the failure scenarios we must tolerate?
  • What is the expected data retention period?
  • Are there any latency SLAs for user-facing operations?
Out of Scope
✗ Designing specific application features (e.g., chat UI)
✗ Security protocols beyond basic authentication
✗ Detailed network infrastructure design
✗ Single-machine optimization techniques
📌
Assumptions
  • 2 billion DAU
  • Each user generates 40 messages per day
  • Average message size is 100 bytes
  • Data retention period is 30 days
⚡
QPS Calculation
write qps2,000,000,000 × 40 / 86,400 ≈ 925,926 writes/sec
peak write qpsAssuming 2× peak factor: 925,926 × 2 ≈ 1,851,852 writes/sec
read qpsAssuming 5 reads per write: 925,926 × 5 ≈ 4,629,630 reads/sec
💾
Storage Estimate
daily storage2,000,000,000 × 40 × 100 bytes = 8,000,000,000,000 bytes ≈ 7.28 TB/day
retention storage7.28 TB × 30 days = 218.4 TB
with replication218.4 TB × 3 replicas = 655.2 TB total storage
🖥️
Server Estimate
write pathAssuming each Cassandra node handles 10,000 writes/sec, need 185 nodes
read pathAssuming each cache node (Redis) handles 50,000 reads/sec, need 93 nodes
🏗️
Architecture Diagram
Architecture Diagram
          +-------------+
          |   Clients   |
          +------+------+
                 |
          +------+-------+
          |  Load Balancer|
          +------+-------+
                 |
    +------------+-------------+
    |                          |
 +--v--+                   +---v---+
 |API  |                   |Cache  |
 |Srv  |                   |(Redis)|
 +--+--+                   +---+---+
    |                          |
    |                          |
 +--v--------------------------v--+
 |       Distributed Database      |
 |        (Apache Cassandra)       |
 +--------------------------------+
                 |
          +------+-------+
          |  Message Queue|
          |   (Apache Kafka)|
          +------+-------+
                 |
          +------+-------+
          | Background   |
          | Workers      |
          +--------------+
🧩
Components
Load Balancer
Distributes incoming client requests evenly across API servers
Why thisNginx or AWS ELB provides high throughput and health checks for availability
Why notDNS round-robin lacks health checks and fine-grained load distribution
API Server
Handles client requests, performs business logic, and interacts with cache and DB
Why thisStateless REST/gRPC servers scale horizontally and allow easy deployment
Why notMonolithic servers cannot scale independently or handle failures gracefully
Cache (Redis Cluster)
Provides low-latency reads for hot data and reduces DB load
Why thisRedis supports sharding and TTL for efficient cache invalidation at scale
Why notMemcached lacks persistence and advanced data structures needed for complex caching
Distributed Database (Apache Cassandra)
Stores large volumes of data with high write throughput and fault tolerance
Why thisCassandra's masterless architecture supports linear write scalability and multi-datacenter replication
Why notPostgreSQL requires manual sharding and does not scale writes linearly at 1.8M writes/sec
Message Queue (Apache Kafka)
Decouples write path from background processing and enables asynchronous workflows
Why thisKafka supports high throughput, partitioning, and durable message storage
Why notRabbitMQ has lower throughput and less efficient partitioning for 1M+ msg/sec
Background Workers
Process asynchronous tasks such as notifications, analytics, and data enrichment
Why thisKubernetes-managed stateless workers allow easy scaling and fault recovery
Why notSingle-threaded cron jobs cannot handle high volume or parallelism
🔄
Data Flow
1
Clients → Load Balancer
Send request
Load balancer evenly distributes requests to API servers
2
Load Balancer → API Server
Forward request
API server handles business logic and checks cache
3
API Server → Cache (Redis)
Read hot data
Cache hit returns data quickly; miss triggers DB read
4
API Server → Distributed Database (Cassandra)
Read/write persistent data
Writes go directly to Cassandra for durability
5
API Server → Message Queue (Kafka)
Publish event
Asynchronous processing decouples heavy tasks
6
Message Queue (Kafka) → Background Workers
Consume event
Workers process notifications, analytics, etc.
7
Background Workers → Distributed Database (Cassandra)
Update derived data
Workers write back processed results

Practice

(1/5)
1. In a system trace, at step 4 the circuit breaker checks the failure count before forwarding a request. What happens if the failure count exceeds the threshold at this step?
easy
A. The circuit breaker opens and immediately returns an error without calling downstream
B. The request is forwarded but logged for monitoring
C. The circuit breaker retries the request twice before opening
D. The request is queued until the downstream service recovers

Solution

  1. Step 1: Trace circuit breaker decision at Step 4

    When failure count exceeds threshold, the circuit breaker transitions to OPEN state and stops forwarding requests.

  2. Step 2: Understand behavior on open circuit

    Requests are immediately failed (B), not forwarded (A), retried (C), or queued (D).

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Open circuit returns error immediately without downstream call [check]
Hint: Open circuit returns error immediately, no downstream call [OK]
Common Mistakes:
2. A search system must provide instant prefix matches for millions of users typing simultaneously, with minimal latency and no external dependencies. Which data structure is best suited to handle this requirement?
easy
A. Use Elasticsearch with inverted indexes to handle prefix queries efficiently
B. Use a prefix hash map that stores all prefixes as keys for O(1) lookups
C. Use a Trie data structure in-memory to provide fast prefix lookups without network overhead
D. Use a relational database with LIKE queries on indexed columns for prefix matching

Solution

  1. Step 1: Analyze latency and dependency constraints

    Elasticsearch involves network calls and disk I/O, increasing latency and external dependencies, so it's unsuitable for minimal latency prefix lookups.

  2. Step 2: Evaluate data structure suitability

    A Trie provides efficient in-memory prefix matching with O(k) lookup time (k = prefix length), ideal for millions of concurrent users with minimal latency.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Trie offers fast prefix search with no external calls [check]
Hint: Trie = fast prefix search in-memory, no network [check]
Common Mistakes:
3. If the message queue (Kafka) is unavailable during the flash sale, what exactly breaks in the order processing pipeline?
medium
A. Payment service retries indefinitely, causing duplicate charges
B. Order requests block at the API layer, causing client timeouts and lost orders
C. Orders are accepted but payment processing is delayed indefinitely
D. Orders are accepted but inventory updates are lost, causing overselling

Solution

  1. Step 1: Trace Kafka's role in order processing

    Kafka decouples order acceptance from inventory update; if Kafka is down, inventory update messages are lost or not sent.

  2. Step 2: Analyze failure impact

    API layer (B) uses async calls, so requests don't block. Payment retries (D) are independent. Accepting orders without inventory update (C) causes overselling.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Kafka down -> inventory updates lost -> overselling risk [check]
Hint: Kafka outage loses inventory updates causing overselling [check]
Common Mistakes:
4. You need to implement a live news feed with many clients receiving frequent updates but no client-to-server messages. Which technology is best considering scalability and simplicity?
medium
A. Use WebSockets to maintain persistent connections for all clients to push updates.
B. Use Server-Sent Events to push unidirectional updates efficiently with automatic reconnection.
C. Use Long Polling to simulate server push by clients repeatedly requesting updates.
D. Use HTTP/2 server push to send updates without client requests.

Solution

  1. Step 1: Analyze communication pattern

    Live news feed requires unidirectional server-to-client updates with many clients and no client messages.

  2. Step 2: Evaluate technology trade-offs

    Server-Sent Events efficiently support unidirectional streaming with automatic reconnection and lower overhead than WebSockets. WebSockets add complexity and resource usage for no client-to-server messages. Long Polling is less efficient due to repeated HTTP requests. HTTP/2 server push is not widely supported for this use case.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Server-Sent Events optimize unidirectional streaming with scalability [check]
Hint: Unidirectional + many clients -> Server-Sent Events [check]
Common Mistakes:
5. At 10x scale, which component of the distributed task scheduler is most likely to become the bottleneck first, and why?
hard
A. The network load balancer, as it cannot handle the increased connection count
B. The cron scheduler, because it cannot generate enough scheduled events concurrently
C. The worker nodes, because CPU resources are exhausted processing tasks
D. The distributed priority queue, due to contention on priority ordering and lease renewals

Solution

  1. Step 1: Analyze cron scheduler scalability

    Cron schedulers are typically lightweight and can be sharded or distributed; unlikely to bottleneck first (A).

  2. Step 2: Analyze worker node resource limits

    Workers scale horizontally; CPU exhaustion (C) is mitigated by adding nodes.

  3. Step 3: Analyze network load balancer capacity

    Load balancers scale with standard techniques; connection count (D) is manageable.

  4. Step 4: Analyze distributed priority queue bottlenecks

    Priority queue faces contention on ordering, lease renewals, and coordination overhead, which grows non-linearly with scale, causing it to bottleneck first (B).

  5. Final Answer:

    Option D -> Option D

  6. Quick Check:

    Priority queue contention and lease overhead limit scale first [check]
Hint: Priority queue contention limits scale before others [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard