Cache stampede prevention in Redis - Deep Dive

Overview - Cache stampede prevention

What is it?

Cache stampede prevention is a technique used to stop many users or processes from trying to refresh the same cached data at the same time. When cached data expires, without prevention, many requests can flood the database or backend to get fresh data, causing slowdowns or crashes. This concept helps manage cache expiration smartly to keep systems fast and stable. It is especially important in systems using Redis or other caching tools.

Why it matters

Without cache stampede prevention, when cached data expires, many users might request the same data simultaneously, overwhelming the database or backend. This can cause slow response times, crashes, or downtime, hurting user experience and business operations. Preventing stampedes keeps systems reliable and fast, even under heavy load.

Where it fits

Before learning cache stampede prevention, you should understand basic caching concepts and how Redis stores and retrieves data. After this, you can learn about advanced caching strategies like cache warming, cache invalidation, and distributed locking to further improve system performance.

Mental Model

Core Idea

Cache stampede prevention ensures only one process refreshes expired cache data while others wait, avoiding overload.

Think of it like...

Imagine a popular restaurant with limited seats. When the menu changes, only one chef updates the menu board while others wait, so customers don’t get confused or overwhelmed by many changes at once.

┌───────────────┐
│ Cache expires │
└──────┬────────┘
       │
       ▼
┌───────────────┐       ┌───────────────┐
│ First request │──────▶│ Refresh cache │
└───────────────┘       └───────────────┘
       │                      ▲
       │                      │
       ▼                      │
┌───────────────┐             │
│ Other requests│─────────────┘
│ wait or use   │
│ old cache     │
└───────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding Basic Caching

Concept: Introduce what caching is and why it is used to speed up data access.

Caching stores copies of data so future requests can be served faster without hitting the main database every time. Redis is a popular tool for caching because it stores data in memory for quick access.

Result

You know that caching helps reduce load on databases and speeds up applications.

Understanding caching is essential because cache stampede prevention builds on how caches work and expire.

2

Foundation

What Causes Cache Stampedes

Concept: Explain how many requests at once can overload the system when cache expires.

When cached data expires, many users might try to get fresh data simultaneously. This floods the database with requests, causing slowdowns or crashes. This problem is called a cache stampede.

Result

You recognize that cache expiration can cause sudden spikes in database load.

Knowing the problem helps you appreciate why prevention techniques are necessary.

3

Intermediate

Using Locks to Prevent Stampedes

🤔Before reading on: do you think a simple lock can fully solve cache stampedes or just reduce them? Commit to your answer.

Concept: Introduce locking mechanisms to allow only one process to refresh cache at a time.

A common method is to use a lock in Redis. When cache expires, the first process acquires the lock and refreshes the cache. Other processes wait or use old cache until the lock is released. This reduces simultaneous database hits.

Result

Only one process refreshes cache, preventing overload.

Understanding locking shows how coordination between processes avoids stampedes.

4

Intermediate

Implementing Cache Expiry with Randomness

🤔Before reading on: do you think setting the same expiry time for all cache keys is better or worse for preventing stampedes? Commit to your answer.

Concept: Explain how adding random expiry times spreads out cache refreshes.

Instead of all cache keys expiring at the same time, add a small random time to each expiry. This staggers cache refreshes, so not all requests hit the database simultaneously.

Result

Cache refreshes happen at different times, reducing load spikes.

Knowing how randomness staggers expiry helps prevent many requests from clustering.

5

Intermediate

Using 'Early Rebuild' or 'Soft Expiry'

🤔Before reading on: do you think serving stale cache while rebuilding is risky or beneficial? Commit to your answer.

Concept: Introduce the idea of serving slightly old cache while refreshing in the background.

Instead of waiting for cache to expire fully, serve stale data and refresh cache in the background. This keeps users happy with fast responses and avoids stampedes.

Result

Users get fast responses even during cache refresh.

Understanding soft expiry balances freshness and performance.

6

Advanced

Distributed Locks and Redlock Algorithm

🤔Before reading on: do you think a single Redis instance lock is enough for distributed systems? Commit to your answer.

Concept: Explain advanced locking for distributed systems using Redlock.

In systems with multiple Redis servers, a single lock might fail if a server crashes. Redlock algorithm uses multiple Redis instances to create a reliable distributed lock, ensuring only one process refreshes cache safely.

Result

Locks work correctly even in complex distributed environments.

Knowing distributed locking prevents rare but serious cache stampede failures.

7

Expert

Handling Failures and Race Conditions

🤔Before reading on: do you think cache stampede prevention is foolproof or can still fail under some conditions? Commit to your answer.

Concept: Discuss edge cases like lock expiration, process crashes, and race conditions.

Locks can expire too soon or processes can crash while holding locks, causing multiple refreshes or stale data. Experts design fallback mechanisms like retry logic, lock renewal, or double-checking cache after lock release to handle these.

Result

Cache stampede prevention is robust and reliable in production.

Understanding failure modes helps build resilient cache systems that avoid hidden bugs.

Under the Hood

Cache stampede prevention works by coordinating processes through locks or timing strategies so only one process queries the backend to refresh cache. Redis stores lock keys with expiration to avoid deadlocks. Random expiry times spread load over time. Soft expiry serves stale data while refreshing asynchronously. Distributed locks use consensus across Redis nodes to ensure safety.

Why designed this way?

Cache stampede prevention was designed to solve the problem of sudden load spikes that degrade system performance. Early caching systems did not handle simultaneous cache misses well, causing outages. Using locks and timing strategies balances freshness, performance, and reliability. Distributed locks address challenges in multi-node environments where single locks can fail.

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│ Cache expires │──────▶│ Acquire lock  │──────▶│ Refresh cache │
└──────┬────────┘       └──────┬────────┘       └──────┬────────┘
       │                       │                       │
       │                       │                       ▼
       │                       │               ┌───────────────┐
       │                       │               │ Release lock  │
       │                       │               └───────────────┘
       │                       │                       │
       ▼                       ▼                       ▼
┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│ Other requests│◀──────│ Wait or use   │◀──────│ Serve fresh   │
│ wait or use   │       │ stale cache   │       │ cache         │
│ stale cache   │       └───────────────┘       └───────────────┘
└───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does setting a single fixed cache expiry time prevent stampedes? Commit yes or no.

Common Belief: If you set a fixed expiry time for cache, stampedes won't happen because all data expires predictably.

Reality: Fixed expiry causes many cache keys to expire simultaneously, leading to stampedes as many requests refresh at once.

Why it matters: Using fixed expiry without randomness can cause sudden load spikes and system slowdowns.

Quick: Is a simple Redis lock always safe in distributed systems? Commit yes or no.

Common Belief: A single Redis lock key is enough to prevent stampedes in any system.

Reality: In distributed systems, single locks can fail due to network issues or crashes, causing multiple refreshes.

Why it matters: Relying on single locks can cause rare but severe cache stampedes in production.

Quick: Does serving stale cache always harm user experience? Commit yes or no.

Common Belief: Serving stale cache is bad and should be avoided to keep data fresh.

Reality: Serving slightly stale cache during refresh keeps responses fast and prevents stampedes, often improving user experience.

Why it matters: Avoiding stale cache can cause slow responses or failures during refresh.

Quick: Can cache stampede prevention guarantee zero backend load spikes? Commit yes or no.

Common Belief: Cache stampede prevention completely eliminates all backend load spikes.

Reality: It reduces but does not fully eliminate spikes; careful design and monitoring are still needed.

Why it matters: Overconfidence can lead to neglecting monitoring and fallback plans.

Expert Zone

1

Lock expiration time must be carefully chosen to avoid premature release or long blocking, balancing safety and performance.

2

Randomized expiry intervals should be tuned to system load patterns to effectively spread cache refreshes without causing stale data issues.

3

Distributed locks like Redlock require careful implementation and understanding of network partitions to avoid rare but critical failures.

When NOT to use

Cache stampede prevention is less useful for data that changes very frequently or is user-specific with low shared cache hits. In such cases, direct database queries or other caching strategies like write-through caches or CDN edge caching may be better.

Production Patterns

In production, teams combine locking with soft expiry and random TTLs. They monitor cache hit rates and backend load, use distributed locks for multi-node Redis, and implement fallback logic for lock failures. Some use libraries or middleware that handle these patterns automatically.

Connections

Distributed Locking

Cache stampede prevention uses distributed locking to coordinate cache refreshes safely across multiple servers.

Understanding distributed locking principles helps design robust cache stampede prevention in complex systems.

Load Balancing

Both cache stampede prevention and load balancing aim to spread workload evenly to avoid overload.

Knowing load balancing concepts clarifies why spreading cache expiry times reduces spikes.

Traffic Shaping in Networking

Cache stampede prevention is similar to traffic shaping, controlling request flow to prevent congestion.

Recognizing this connection helps apply network traffic control ideas to caching strategies.

Common Pitfalls

#1 Not using locks causes multiple processes to refresh cache simultaneously.

Wrong approach:

if (!cache.exists(key)) {
data = fetchFromDB();
cache.set(key, data);
}
return cache.get(key);

Correct approach:

if (acquireLock(key + ':lock')) {
data = fetchFromDB();
cache.set(key, data);
releaseLock(key + ':lock');
}
return cache.get(key);

Root cause:

Misunderstanding that cache misses can happen concurrently, causing many refreshes.

#2 Setting the same expiry time for all cache keys leads to synchronized expiration.

Wrong approach:

cache.set(key, data, expire=3600); // fixed 1 hour expiry

Correct approach:

cache.set(key, data, expire=3600 + random(0,300)); // add random 0-5 min

Root cause:

Not realizing that identical expiry times cause many keys to expire together.

#3 Using locks without expiration can cause deadlocks if process crashes.

Wrong approach:

setLock(key + ':lock'); // no expiry set

Correct approach:

setLock(key + ':lock', expire=10); // lock expires after 10 seconds

Root cause:

Forgetting to set lock expiration leads to permanent lock if process fails.

Key Takeaways

Cache stampede prevention stops many processes from refreshing cache simultaneously, protecting backend systems.

Using locks, random expiry times, and soft expiry are key strategies to prevent stampedes effectively.

Distributed locking is essential in multi-node Redis setups to avoid rare but serious failures.

Understanding failure modes and tuning parameters carefully ensures robust and reliable cache systems.

Cache stampede prevention balances data freshness, system performance, and user experience in real-world applications.

Practice

1. What is the main goal of cache stampede prevention in Redis?

easy

A. To allow multiple processes to update cache simultaneously

B. To increase the size of the cache automatically

C. To delete all cache entries periodically

D. To stop many requests from hitting the database at the same time

Solution

1. Step 1: Understand cache stampede problem

   Cache stampede happens when many requests try to refresh expired cache simultaneously, causing database overload.

2. Step 2: Identify the goal of prevention

   Prevention aims to allow only one process to refresh cache, reducing database hits.

3. Final Answer:

   To stop many requests from hitting the database at the same time -> Option D

4. Quick Check:

   Cache stampede prevention = stop many DB hits [OK]

Hint: Cache stampede means many requests hit DB; prevention stops this [OK]

Common Mistakes:

• Thinking cache stampede increases cache size
• Believing cache stampede deletes cache entries
• Assuming multiple processes update cache simultaneously

2. Which Redis command syntax correctly sets a lock with expiration to prevent cache stampede?

easy

A. SET key value NX PX 3000

B. SET key value XX EX 3000

C. GET key NX PX 3000

D. DEL key NX PX 3000

Solution

1. Step 1: Recall Redis SET options for locking

   SET with NX sets key only if it does not exist; PX sets expiration in milliseconds.

2. Step 2: Identify correct syntax

   SET key value NX PX 3000 uses SET with NX and PX correctly to create a lock with 3000ms expiry.

3. Final Answer:

   SET key value NX PX 3000 -> Option A

4. Quick Check:

   SET with NX and PX = lock with expiry [OK]

Hint: Use SET with NX and PX for lock with expiry [OK]

Common Mistakes:

• Using GET instead of SET for locking
• Using XX instead of NX option
• Using DEL command for locking

3. Given this Redis Lua script snippet for cache stampede prevention:

local lock = redis.call('SET', KEYS[1], '1', 'NX', 'PX', 5000)
if not lock then
  return redis.error_reply('Lock not acquired')
end
-- refresh cache here
redis.call('DEL', KEYS[1])
return 'Cache refreshed'

What will be the output if the lock key already exists?

medium

A. Lock not acquired

B. nil

C. Syntax error

D. 'Cache refreshed'

Solution

1. Step 1: Understand SET with NX behavior

   SET with NX returns OK if key set, nil if key exists already.

2. Step 2: Check script logic when lock not acquired

   If lock is nil (not acquired), script returns redis.error_reply('Lock not acquired').

3. Final Answer:

   Lock not acquired -> Option A

4. Quick Check:

   Lock exists -> error reply 'Lock not acquired' [OK]

Hint: If SET NX fails, script returns 'Lock not acquired' error [OK]

Common Mistakes:

• Assuming script returns 'Cache refreshed' always
• Expecting nil instead of error reply
• Thinking script has syntax error

4. You wrote this Redis command to prevent cache stampede:

SET lock_key 1 PX NX 5000

But it does not work as expected. What is the problem?

medium

A. The value '1' is invalid for locking

B. The order of PX and NX options is incorrect

C. SET command does not support NX option

D. PX option requires seconds, not milliseconds

Solution

1. Step 1: Check Redis SET command option order

   Redis requires SET options in order: NX before PX or PX before NX is accepted, but best practice is NX PX.

2. Step 2: Identify issue with given command

   Given command uses PX before NX, which Redis does not accept; correct order is NX PX.

3. Final Answer:

   The order of PX and NX options is incorrect -> Option B

4. Quick Check:

   Correct option order is NX PX [OK]

Hint: Use NX before PX in SET command options [OK]

Common Mistakes:

• Thinking value '1' is invalid
• Believing SET does not support NX
• Confusing PX milliseconds with seconds

5. You want to prevent cache stampede using Redis locks. Which approach correctly handles the scenario where the cache refresh takes longer than the lock expiration time?

hard

A. Do not use PX expiration and rely on manual DEL of lock key only

B. Set a very short PX expiration to release lock quickly and retry immediately

C. Set a longer PX expiration than the expected refresh time and renew the lock if needed

D. Use GETSET command to overwrite lock without expiration

Solution

1. Step 1: Understand lock expiration vs refresh time

   If lock expires before refresh finishes, another process may acquire lock causing duplicate refresh.

2. Step 2: Choose approach to avoid premature lock expiration

   Setting PX longer than refresh time and renewing lock prevents other processes from acquiring lock too early.

3. Final Answer:

   Set a longer PX expiration than the expected refresh time and renew the lock if needed -> Option C

4. Quick Check:

   Lock expiry > refresh time + renew if needed [OK]

Hint: Lock expiry must cover refresh duration; renew lock if needed [OK]

Common Mistakes:

• Setting too short PX causing multiple refreshes
• Not using PX expiration causing stale locks
• Using GETSET which overwrites lock unsafely