Bird
Raised Fist0

Why System Design Exists and What Single Machines Cannot Do

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🎯
Why System Design Exists and What Single Machines Cannot Do
45 min · senior
senior

You are an engineering lead at a startup with 50 million daily active users (DAU). The product requires real-time data processing and low-latency responses under heavy load. Single machines cannot handle the scale, availability, and fault tolerance needed. Design a system that addresses these challenges while meeting strict constraints.

Constraints
  • End-to-end encryption must be maintained without sacrificing performance
  • System must support offline delivery and eventual consistency
  • P99 latency must be under 200ms for user-facing requests
  • Data durability and fault tolerance across multiple data centers
  • Support 50M DAU with peak throughput of 1M requests per second
📋
Guided Steps
1/5
Step 1
Choosing the Data Ingestion and Messaging Layer
How would you design the ingestion pipeline to handle 1M requests/sec with ordering guarantees?
💡 Hint
Consider partitioning keys and durable distributed messaging systems that support ordering.
Key Decision
Use Apache Kafka with topic partitioning keyed by user_id to ensure ordered processing per user and linear scalability.
Interviewer Expectation
Mid: Names Kafka for messaging. Senior: Explains user_id partition key ensures all messages for a user go to the same partition, preserving order and enabling consumer affinity.
⚠️Common Wrong Answer
Using RabbitMQ without partitioning - leads to unordered message delivery and bottlenecks at high throughput.
Step 2
Data Storage for High Throughput and Low Latency
What storage system would you choose to store user data with fast writes and range queries for synchronization?
💡 Hint
Look for a distributed NoSQL database optimized for write-heavy workloads and efficient range scans.
Key Decision
Use Apache Cassandra with partition key as user_id to enable O(1) writes and efficient cursor-based sync via clustering columns.
Interviewer Expectation
Mid: Names Cassandra for scalability. Senior: Explains why user_id as partition key ensures data locality and why clustering columns enable efficient range queries for incremental sync.
⚠️Common Wrong Answer
Using a relational database with sharding - causes complex joins and poor write scalability at 1M writes/sec.
Step 3
Ensuring End-to-End Encryption and Security
How do you maintain end-to-end encryption without degrading system performance?
💡 Hint
Consider client-side encryption and zero-trust principles to avoid decrypting data in transit or at rest on servers.
Key Decision
Implement client-side encryption with keys managed by the client; servers store and route encrypted blobs without decrypting, using TLS for transport security.
Interviewer Expectation
Mid: Mentions TLS for transport security. Senior: Details client-side encryption to prevent server-side decryption, preserving privacy and reducing attack surface.
⚠️Common Wrong Answer
Encrypting only at rest on servers - exposes data during transit and processing, violating end-to-end encryption.
Step 4
Handling Offline Delivery and Eventual Consistency
How would you design the system to support offline clients and ensure eventual consistency upon reconnection?
💡 Hint
Think about durable message queues and client-side state reconciliation with conflict resolution strategies.
Key Decision
Use Kafka for durable message storage with consumer offsets for replay; clients sync using cursor-based APIs and resolve conflicts via vector clocks or CRDTs.
Interviewer Expectation
Mid: Describes durable queues and replay. Senior: Explains cursor-based sync with Kafka offsets and conflict resolution using CRDTs for convergent state.
⚠️Common Wrong Answer
Stateless HTTP polling without durable queues - causes message loss and inconsistent client state after offline periods.
Step 5
Meeting P99 Latency Under 200ms
What architectural choices ensure P99 latency under 200ms at peak load?
💡 Hint
Consider caching, load balancing, and asynchronous processing to reduce tail latency.
Key Decision
Use Redis as a read-through cache for hot user data, NGINX load balancers with consistent hashing, and asynchronous Kafka consumers to decouple ingestion from processing.
Interviewer Expectation
Mid: Mentions caching and load balancing. Senior: Explains Redis TTL caching reduces DB hits, consistent hashing preserves cache locality, and async processing prevents blocking on writes.
⚠️Common Wrong Answer
Direct DB queries on every request - causes high tail latency and DB overload at scale.
🔌
API Design Challenge
Sketch 3 core API endpoints that enable client interaction with the system at scale.
Hints
  • How does the client send data securely and efficiently?
  • How does the client synchronize state after offline periods?
  • How does the client acknowledge receipt to ensure delivery guarantees?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Use of partitioned distributed messaging system (Kafka) with appropriate partition key
  • Choice of scalable NoSQL database (Cassandra) with data model supporting range queries
  • Implementation of client-side encryption to maintain end-to-end security
  • Design for offline delivery with durable queues and sync mechanisms
  • Architectural decisions to meet strict P99 latency requirements
Good to Have - 30%
  • Use of caching layer (Redis) to reduce latency
  • Load balancing strategy preserving cache locality (NGINX consistent hashing)
  • Conflict resolution strategy for eventual consistency (CRDTs or vector clocks)
  • Multi-region replication for fault tolerance
Bonus - 10%
  • Detailed API design addressing encryption, sync, and acknowledgments
  • Discussion of failure modes and mitigation strategies
  • Trade-off analysis between consistency, availability, and partition tolerance
Anti-patterns to Avoid
Using a single machine or monolithic database causing bottlenecks and single points of failure, which leads to system crashes and inability to scale.
Stateless HTTP polling for message delivery leading to excessive load and latency spikes, causing system collapse under high user concurrency.
Server-side decryption violating end-to-end encryption guarantees and increasing attack surface, exposing sensitive data to internal threats.
Relational databases with complex joins causing poor write scalability at high throughput, resulting in slow response times and bottlenecks.
Ignoring offline client scenarios resulting in data loss and inconsistent state, degrading user experience and data integrity.
Direct DB queries on every request without caching causing high tail latency and overload, leading to system instability under peak load.
✅
Model Answer
▼ Reveal

The system uses Apache Kafka with user_id partitioning for ordered ingestion, Apache Cassandra for scalable storage with efficient range queries, and client-side encryption to maintain end-to-end security. Offline delivery is supported via durable Kafka queues and cursor-based sync APIs with conflict resolution using CRDTs or vector clocks. Redis caching and NGINX load balancers with consistent hashing ensure P99 latency under 200ms. Multi-region deployment provides durability and fault tolerance.

Key Decisions
  • Use Kafka with user_id partition key to guarantee ordered message ingestion per user and scale to 1M requests/sec
  • Choose Cassandra for write-heavy workloads with efficient range queries via clustering columns for incremental sync
  • Implement client-side encryption to maintain end-to-end encryption without server-side decryption, using TLS for transport security
  • Support offline clients with durable Kafka queues and cursor-based sync APIs, resolving conflicts with CRDTs or vector clocks
  • Employ Redis as a read-through cache with TTL to reduce latency and load on Cassandra
  • Use NGINX load balancers with consistent hashing to maintain session affinity and cache locality
  • Deploy multi-region replication for data durability and fault tolerance across data centers
  • Apache Kafka with user_id partition key for ordered, scalable message ingestion
  • Apache Cassandra with user_id partition key and clustering columns for efficient writes and range queries
  • Client-side encryption with TLS transport to maintain end-to-end encryption without server decryption
  • Kafka consumer offsets and cursor-based sync APIs to support offline delivery and eventual consistency
  • Redis read-through cache with TTL to reduce latency and load on primary storage
  • NGINX load balancers with consistent hashing to maintain session affinity and cache locality
  • Conflict resolution using CRDTs or vector clocks for convergent client state after offline periods
  • Multi-region deployment with data replication for durability and fault tolerance

Practice

(1/5)
1. A web API needs to enforce a rate limit of 100 requests per minute per user, allowing bursts but avoiding sudden resets at minute boundaries. Which rate limiting approach best fits this requirement?
easy
A. Fixed window counter, resetting counts exactly every minute
B. Sliding window counter, tracking requests over the last 60 seconds continuously
C. Token bucket with fixed refill intervals every minute
D. Leaky bucket with fixed drain rate per minute

Solution

  1. Step 1: Identify burst tolerance and smooth rate enforcement

    Fixed window counters reset abruptly at minute boundaries, causing spikes. Token and leaky buckets enforce steady rates but may not track exact recent usage.

  2. Step 2: Sliding window counters track requests continuously over the last 60 seconds, allowing bursts without sudden resets

    Sliding window counters provide smoother rate limiting by considering recent activity, avoiding boundary effects.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Sliding window smooths rate limits over time, avoiding fixed window spikes [check]
Hint: Sliding window smooths counts over time, fixed resets abruptly [check]
Common Mistakes:
2. A public API receives millions of requests per minute from diverse clients. To prevent abuse, which layer should primarily enforce a global rate limit that blocks excessive requests before reaching internal services?
easy
A. Apply rate limiting exclusively inside each microservice to control business logic
B. Implement rate limiting only at the database layer to prevent overload
C. Enforce rate limiting at the API Gateway to block abusive traffic early
D. Use client-side rate limiting to reduce server load

Solution

  1. Step 1: Identify the role of API Gateway

    The API Gateway acts as the first entry point and can block abusive traffic before it reaches internal services or databases.

  2. Step 2: Evaluate other layers

    Database-level rate limiting is too late and risks overload; service-level limits are per-service and don't prevent global abuse; client-side limits are unreliable as clients can be malicious.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    API Gateway blocks abusive requests early [check]
Hint: Block abusive traffic earliest at API Gateway [check]
Common Mistakes:
3. In the search ranking system request flow, at step 3 the Ranking Component calculates BM25 scores for candidate documents. What happens if the document frequency (DF) data is stale at this step?
easy
A. The query is rejected until fresh DF data is available
B. BM25 scores are recalculated later with fresh DF, so initial ranking is unaffected
C. The system falls back to TF-IDF scoring automatically to avoid stale data impact
D. Ranking Component uses stale DF, causing inaccurate relevance scores and suboptimal ranking

Solution

  1. Step 1: Understand BM25 reliance on DF

    BM25 scoring depends on accurate document frequency to weigh term rarity; stale DF skews scores.

  2. Step 2: Trace impact of stale DF at Ranking Component

    Ranking uses stale DF directly, leading to inaccurate relevance scores and suboptimal document ordering; no automatic fallback or query rejection occurs.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Stale DF causes inaccurate BM25 scores and ranking errors [check]
Hint: BM25 depends on fresh DF; stale DF skews scores [check]
Common Mistakes:
4. In the Ticket Server pattern request flow, at Step 3 the Ticket Server allocates an ID block to a client. What state does the client hold immediately after Step 3 before using any IDs?
easy
A. Client holds a reserved ID range (block) ready for local allocation
B. Client holds a single ID and waits for confirmation before next request
C. Client holds no IDs and must request one ID per operation synchronously
D. Client holds a timestamp token to generate IDs locally without server contact

Solution

  1. Step 1: Trace Ticket Server allocation at Step 3

    The Ticket Server allocates a block of IDs (a range) to the client to reduce coordination.

  2. Step 2: Understand client state post-allocation

    Client holds this reserved ID block locally, enabling fast local ID generation without further immediate server requests.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Client holds ID block -> local allocation without blocking [check]
Hint: Client caches ID blocks locally to avoid per-ID server calls [check]
Common Mistakes:
5. If the Market Data Feed component crashes and stops publishing updates, what is the exact failure observed by external clients subscribing to real-time stock prices?
medium
A. Clients receive stale prices indefinitely with no indication of failure
B. Clients receive partial updates but with delayed timestamps causing confusion
C. Clients receive an error message and automatic reconnection attempts begin
D. Clients continue receiving cached data from the Matching Engine without updates

Solution

  1. Step 1: Understand Market Data Feed failure impact

    When Market Data Feed crashes, it stops pushing updates; clients detect connection loss and receive error events triggering reconnection.

  2. Step 2: Eliminate other options

    Clients receive stale prices indefinitely with no indication of failure is wrong because clients detect connection loss; Clients receive partial updates but with delayed timestamps causing confusion is wrong because partial updates don't occur without feed; Clients continue receiving cached data from the Matching Engine without updates is wrong because Matching Engine doesn't serve cached market data.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Feed crash -> clients get error + reconnect attempts [check]
Hint: Feed crash triggers client error and reconnect [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard