Bird
Raised Fist0

Latency vs Throughput - The Core Trade-off Every Design Must Balance

HLDmediumInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/6
API Gateway

Receives client request and enqueues message

Transforms WebSocket request into Kafka message

Protocol:WebSocket
BeforeClient request is received by API Gateway
AfterMessage is published to Kafka Cluster for durability
Kafka Cluster

Stores message durably and queues for processing

Partitions message for throughput and ordering

Protocol:Kafka
BeforeMessage is published by API Gateway
AfterMessage is durably stored and ready for consumption
Delivery Service

Consumes message from Kafka and checks cache

Queries Cache Layer for recent data to reduce latency

Protocol:Kafka Consumer / Redis GET
BeforeMessage is available in Kafka for processing
AfterDelivery Service has cache hit/miss result for message
Cache Layer

Returns cached data if available

Responds to Delivery Service with cached content or miss

Protocol:Redis GET
BeforeDelivery Service requests cached data
AfterCache Layer returns data or indicates cache miss
Delivery Service

Fetches data from Cassandra DB if cache miss

Queries Cassandra DB for durable data retrieval

Protocol:CQL Query
BeforeCache Layer indicates cache miss
AfterDelivery Service obtains durable data from Cassandra DB
Cassandra DB

Returns requested data to Delivery Service

Serves query with eventual consistency guarantees

Protocol:CQL Response
BeforeDelivery Service issues query to Cassandra DB
AfterData is returned to Delivery Service for delivery
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────┐
│  API Gateway  │
└───────┬───────┘
        │ WebSocket
        ▼
┌───────────────┐
│ Kafka Cluster │
└───────┬───────┘
        │ Kafka
        ▼
┌───────────────┐
│ Delivery Srv  │
└───────┬───────┘
        │ Redis GET
        ▼
┌───────────────┐
│ Cache Layer   │
└───────┬───────┘
        │ CQL Query
        ▼
┌───────────────┐
│ Cassandra DB  │
└───────────────┘
Design Rationale

This architecture illustrates the core trade-off between latency and throughput. The API Gateway accepts client requests and immediately enqueues them into Kafka, enabling high throughput by decoupling request intake from processing. Kafka ensures durable, ordered message storage but adds latency due to asynchronous processing. The Delivery Service consumes messages from Kafka and attempts to serve data from the Cache Layer to minimize latency for hot data. On cache misses, it queries Cassandra DB, which provides durable storage but with higher read latency. This layered approach balances throughput and latency: Kafka and Cassandra optimize for throughput and durability, while the Cache Layer reduces latency for frequent reads. Understanding this flow helps engineers design systems that balance these competing demands effectively.

Primary Focus

✓

Practice

(1/5)
1. A distributed caching system needs to assign keys to cache nodes such that minimal remapping occurs when nodes join or leave. Which component best handles this requirement?
easy
A. Use Rendezvous Hashing to assign keys to nodes with highest hash score
B. Use a simple modulo-based hash function on node count
C. Use a centralized directory service mapping keys to nodes
D. Use consistent hashing with virtual nodes and a hash ring

Solution

  1. Step 1: Understand minimal remapping requirement

    Modulo-based hashing (Use a simple modulo-based hash function on node count) causes massive remapping when node count changes. Centralized directory (Use a centralized directory service mapping keys to nodes) adds single point of failure and latency. Consistent hashing with virtual nodes (Use consistent hashing with virtual nodes and a hash ring) reduces remapping but is more complex and less direct than Rendezvous Hashing.

  2. Step 2: Identify Rendezvous Hashing benefits

    Rendezvous Hashing assigns keys to nodes by computing scores and picking the highest, minimizing remapping and avoiding ring maintenance.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Rendezvous Hashing picks highest score node -> minimal remapping [check]
Hint: Highest hash score wins key assignment [check]
Common Mistakes:
2. If the Kafka cluster responsible for video upload event streaming goes down, what exactly breaks in the upload processing pipeline?
medium
A. Users cannot upload videos because the frontend blocks until Kafka responds
B. Video metadata extraction and transcoding jobs stall, causing delayed availability of videos
C. The For-You feed algorithm stops recommending new videos immediately
D. The CDN fails to cache videos, causing playback failures

Solution

  1. Step 1: Trace Kafka's role in upload pipeline

    Kafka buffers upload events for async processing; frontend uploads don't block on Kafka (A wrong).

  2. Step 2: Identify failure impact

    Metadata extraction and transcoding (B) stall but do not cause immediate video unavailability; feed algorithm (D) depends on processed data, not Kafka directly.

  3. Step 3: Recognize CDN caching dependency

    Without Kafka, processed videos don't reach CDN timely, causing cache misses and playback failures (C).

  4. Final Answer:

    Option D -> Option D

  5. Quick Check:

    Kafka outage delays CDN caching, causing playback failures [check]
Hint: Kafka outage stalls processing, causing CDN cache misses [OK]
Common Mistakes:
3. When deciding between a monolithic and microservices architecture during a 45-minute system design interview, which choice best fits a scenario requiring rapid iteration and minimal upfront complexity?
medium
A. Choose monolithic to reduce initial complexity and speed iteration
B. Choose microservices to maximize scalability regardless of timeline
C. Choose microservices because they always improve fault isolation
D. Choose monolithic because it eliminates the need for any deployment automation

Solution

  1. Step 1: Analyze timeline and complexity constraints

    Rapid iteration and minimal upfront complexity favor monolithic design to avoid distributed system overhead.

  2. Step 2: Eliminate options mismatched to constraints

    Microservices add complexity and deployment overhead, unsuitable for quick iteration; monoliths still require deployment automation.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Monolith -> faster iteration, less initial complexity [check]
Hint: Monolith for speed, microservices for scale [check]
Common Mistakes:
4. For a search system with highly skewed term distributions and frequent query updates, which scoring approach is better and why?
medium
A. Use classic TF-IDF because it is simpler and faster to compute
B. Use raw term frequency only to avoid overhead of inverse document frequency
C. Use BM25 because it handles term saturation and document length normalization better
D. Use a vector space model with cosine similarity ignoring term frequency

Solution

  1. Step 1: Analyze term distribution and query update needs

    Skewed term distributions cause TF-IDF to overweight rare terms excessively; frequent updates require robust scoring.

  2. Step 2: Evaluate scoring methods

    BM25 accounts for term frequency saturation and normalizes by document length, improving relevance in skewed data; TF-IDF and raw TF lack these refinements; vector space ignoring TF loses relevance signals.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    BM25 handles skew and length normalization better than TF-IDF [check]
Hint: BM25 normalizes length and term saturation better than TF-IDF [check]
Common Mistakes:
5. For a social media feed requiring low-latency reads and flexible schema, which database choice is better and why?
medium
A. Use a relational database with normalized tables for consistency
B. Use a wide-column NoSQL store to denormalize and optimize read latency
C. Use a graph database to model user relationships for feed generation
D. Use an in-memory cache only to serve all feed requests

Solution

  1. Step 1: Analyze read latency and schema flexibility needs

    Social feeds require fast reads and flexible schema to handle diverse content types.

  2. Step 2: Compare database options

    Relational DBs have join overhead; graph DBs excel at relationships but not high-volume feed reads; in-memory cache alone lacks persistence.

  3. Step 3: Justify wide-column NoSQL choice

    Wide-column stores denormalize data for fast reads and support flexible schemas, fitting feed requirements.

  4. Final Answer:

    Option B -> Option B

  5. Quick Check:

    Wide-column NoSQL -> denormalized, low-latency reads [check]
Hint: Low-latency feed -> denormalized wide-column store [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard