Bird
Raised Fist0

Cache Stampede - Thundering Herd Problem and Prevention Strategies

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/7
Client

Sends request for data

Client issues HTTP request to load balancer

Protocol:HTTP
BeforeClient needs data not locally cached
AfterRequest sent to Load Balancer
Redis Cache

Checks cache for requested key

Load Balancer forwards request; Redis GET returns miss

Protocol:Redis GET
BeforeCache does not contain requested data
AfterCache miss detected; triggers lock acquisition
Lock Manager (Redis-based)

Attempts to acquire distributed lock

Uses Redis SETNX to lock key for DB query

Protocol:Redis SETNX
BeforeNo lock held for this key
AfterLock acquired by first requester; others wait
Backend Data Store (Cassandra)

Queries database for data

Single DB query performed due to lock

Protocol:CQL Query
BeforeNo cached data; lock held
AfterDB returns fresh data for caching
Redis Cache

Populates cache with fresh data

Sets key with TTL to avoid stale data

Protocol:Redis SET with TTL
BeforeCache missing data for key
AfterCache now contains fresh data
Lock Manager (Redis-based)

Releases distributed lock

Deletes lock key allowing others to proceed

Protocol:Redis DEL
BeforeLock held by first requester
AfterLock released; other clients can access cache
Wait & Poll Mechanism

Other clients poll cache for data

Clients retry Redis GET until cache populated

Protocol:Redis GET Polling
BeforeClients blocked waiting for lock release
AfterClients receive cached data without DB hit
🏗️
Architecture Diagram
Architecture Diagram
┌─────────┐
│ Client  │
└────┬────┘
     │ HTTP Request
     ▼
┌───────────────┐
│ Load Balancer │
└────┬──────────┘
     │ Forward HTTP
     ▼
┌───────────────────────────┐
│ Redis Cache               │
│ (Check for cached value)  │
└────┬──────────────────────┘
     │ Redis GET
     ▼
┌───────────────────────────────┐
│ Lock Manager (Redis-based)    │
│ (Acquire lock to prevent stampede) │
└────┬──────────────────────────┘
     │ Redis SETNX (lock attempt)
     ▼
┌───────────────────────────────┐
│ Backend Data Store (Cassandra)│
│ (Single DB query on cache miss)│
└────┬──────────────────────────┘
     │ CQL Query
     ▼
┌───────────────────────────────┐
│ Redis Cache                   │
│ (Populate cache with DB data) │
└────┬──────────────────────────┘
     │ Redis SET with TTL
     ▼
┌───────────────────────────────┐
│ Lock Manager (Redis-based)    │
│ (Release lock)                │
└────┬──────────────────────────┘
     │ Redis DEL (unlock)
     ▼
┌───────────────────────────────┐
│ Wait & Poll Mechanism          │
│ (Other clients poll cache)     │
└───────────────────────────────┘
Design Rationale

This architecture prevents the cache stampede problem by using a Redis-based distributed lock to ensure only one client queries the backend database on a cache miss. When a client requests data, the Redis Cache is checked first. If the data is missing, the Lock Manager attempts to acquire a lock using Redis SETNX. The first client to acquire the lock queries the Backend Data Store and populates the cache with fresh data. Other clients wait and poll the cache via the Wait & Poll Mechanism, avoiding simultaneous DB queries. Once the cache is updated, the lock is released, allowing waiting clients to read from cache directly. This design reduces load on the database and prevents thundering herd effects during cache misses.

Primary Focus

✓

Practice

(1/5)
1. In the Snowflake ID generation flow, at Step 3 the sequence number is incremented after the timestamp and machine ID are set. What happens if the sequence number exceeds its maximum value within the same millisecond?
easy
A. The generator blocks and waits until the next millisecond to reset the sequence number.
B. The system resets the sequence to zero and continues generating IDs immediately.
C. The sequence number wraps around causing duplicate IDs within the same millisecond.
D. The machine ID is incremented to avoid sequence overflow.

Solution

  1. Step 1: Understand sequence number role in Snowflake

    The sequence number differentiates IDs generated within the same millisecond on the same machine.

  2. Step 2: Analyze overflow handling

    If the sequence exceeds its max (e.g., 4095), the generator must wait for the next millisecond to avoid duplicates. Resetting immediately (A) risks duplicates; wrapping (C) causes collisions; incrementing machine ID (D) is not part of the design.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Sequence overflow -> wait next millisecond to avoid duplicates [check]
Hint: Sequence overflow blocks until next millisecond [OK]
Common Mistakes:
2. If the reverse proxy responsible for SSL termination crashes, what exactly breaks in the request flow?
medium
A. Clients can still connect directly to backend servers without interruption
B. Clients' SSL handshakes fail, so connections never reach backend servers
C. Backend servers receive unencrypted traffic, exposing internal data
D. The reverse proxy forwards requests without decrypting, causing backend errors

Solution

  1. Step 1: Understand reverse proxy SSL termination failure

    If the reverse proxy crashes, SSL termination stops, so backend servers receive unencrypted traffic if clients connect directly or fallback occurs.

  2. Step 2: Eliminate other options

    Clients cannot connect directly if proxy is mandatory; SSL handshake fails only if proxy blocks connection; proxy can't forward encrypted requests without decrypting if crashed.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Reverse proxy crash exposes backend to unencrypted traffic [check]
Hint: Reverse proxy crash exposes backend to unencrypted traffic [check]
Common Mistakes:
3. Without the asynchronous replication component in a PACELC system prioritizing latency, what exactly breaks during a network partition?
medium
A. Reads return stale data indefinitely without updates
B. Writes block until all replicas acknowledge, causing high latency
C. Local writes succeed but are lost if the node crashes before replication
D. The system automatically switches to a single primary mode

Solution

  1. Step 1: Identify role of asynchronous replication

    It propagates updates to other replicas to keep data fresh despite partitions.

  2. Step 2: Trace failure without async replication

    Without it, replicas do not get updates, so reads return stale data indefinitely.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    No async replication -> stale reads persist during partition [check]
Hint: No async replication -> stale reads never update during partition [check]
Common Mistakes:
4. At 10x scale, which component in the real-time analytics pipeline is most likely to fail first and why?
hard
A. Kafka brokers, due to partition leader overload and disk I/O bottlenecks
B. Dashboard frontend, as WebSocket connections exceed browser limits
C. Flink cluster, because stateful operators require large memory and checkpointing slows down
D. OLAP store, because complex ad-hoc queries overwhelm its indexing and caching layers

Solution

  1. Step 1: Analyze Kafka broker bottlenecks at scale

    Kafka brokers face partition leader overload and disk I/O saturation as event volume grows 10x.

  2. Step 2: Compare other components

    Flink scales with more nodes; OLAP stores can shard; dashboards scale horizontally; Kafka's broker disk and network limits are hardest to scale first.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Kafka broker disk I/O saturates first at 10x event volume [check]
Hint: Kafka broker disk I/O saturates before others scale [check]
Common Mistakes:
5. At 10x scale, which component is most likely to become the bottleneck first in the typeahead system, and why?
hard
A. The front-end API servers, because they cannot handle the increased number of concurrent connections
B. The personalization data store, because user profile reads increase linearly and cannot be sharded easily
C. The ranking service, because ranking computations grow exponentially with candidate suggestions
D. The Trie cache servers, because maintaining low-latency prefix lookups requires large in-memory state that is hard to shard further

Solution

  1. Step 1: Analyze component scalability

    Personalization stores can be sharded; ranking computations scale linearly; front-end servers can be horizontally scaled.

  2. Step 2: Identify Trie cache bottleneck

    Trie caches hold large in-memory prefix trees; sharding by prefix has limits due to prefix overlap and memory footprint, causing latency and memory bottlenecks.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Trie cache memory and shard limits -> first bottleneck at 10x scale [check]
Hint: Trie cache memory limits cause first bottleneck at scale [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard