Distributed caching (Redis, Memcached) in HLD - System Design Guide

Problem Statement

When a single cache server is used, it becomes a bottleneck under high load, causing slow response times and potential downtime if the cache server fails. Also, a single cache cannot handle the large data volume needed for scalable applications, leading to cache misses and increased load on the database.

Solution

Distributed caching spreads cached data across multiple cache servers, allowing the system to handle more requests and larger data volumes. It uses consistent hashing or partitioning to route requests to the correct cache node, ensuring fast data retrieval and fault tolerance by replicating data or failing over to other nodes.

Architecture

Client App

Client App

↓

Cache Client

Cache Client

↓

Cache 1

(Redis)

↓

Database

Database

This diagram shows a client application using a cache client with consistent hashing to distribute requests across multiple Redis cache nodes, which in turn reduce load on the database.

Trade-offs

✓ Pros

→

Improves cache capacity and throughput by distributing data across multiple nodes.

→

Increases fault tolerance; if one cache node fails, others can serve requests.

→

Reduces latency by routing requests to the correct cache node directly.

→

Enables horizontal scaling of cache layer as demand grows.

✗ Cons

→

Adds complexity in managing cache consistency and invalidation across nodes.

→

Requires careful partitioning and hashing to avoid uneven data distribution (hot spots).

→

Replication and failover mechanisms increase operational overhead.

Use distributed caching when your application has high read traffic exceeding thousands of requests per second and requires low latency with large cache data sets that cannot fit on a single server.

Avoid distributed caching if your cache data is small enough to fit on a single server or if your read traffic is under 1,000 requests per second, where added complexity outweighs benefits.

Real World Examples

Netflix

Uses distributed Redis caching to store user session data and metadata, enabling fast access and fault tolerance across global edge locations.

Twitter

Employs Memcached clusters to cache timelines and user data, reducing database load and improving response times during traffic spikes.

Uber

Uses distributed caching to store frequently accessed geolocation and pricing data, ensuring low latency for ride matching.

Alternatives

Local in-memory caching

Caches data only on the application server's memory without sharing across nodes.

Use when: When the application is small-scale or stateful per instance, and data consistency across nodes is not critical.

CDN caching

Caches static content closer to users at edge locations rather than application data in memory.

Use when: When the main goal is to speed up delivery of static assets like images, videos, or web pages.

Database caching with read replicas

Uses database replicas to distribute read load instead of a separate cache layer.

Use when: When data freshness is critical and caching complexity is to be minimized.

Summary

Distributed caching spreads cached data across multiple servers to handle high traffic and large data volumes.

It improves fault tolerance and reduces latency by routing requests to the correct cache node.

However, it adds complexity in managing data consistency and requires careful partitioning.

Practice

1. What is the primary purpose of using a distributed cache like Redis or Memcached in a system?

easy

A. To reduce database load by storing frequently accessed data in memory

B. To permanently store all application data for backup

C. To replace the database entirely for all data storage needs

D. To increase the size of the database by adding more servers

Solution

1. Step 1: Understand the role of distributed cache

   Distributed caches store frequently accessed data in fast memory to reduce the number of database queries.

2. Step 2: Identify the main benefit

   This reduces the load on the database and speeds up data retrieval for repeated requests.

3. Final Answer:

   To reduce database load by storing frequently accessed data in memory -> Option A

4. Quick Check:

   Distributed cache reduces DB load = C [OK]

Hint: Cache stores frequent data to reduce DB hits [OK]

Common Mistakes:

• Thinking cache permanently stores all data
• Confusing cache with database backup
• Assuming cache replaces the database

2. Which of the following is the correct way to set a key-value pair in Redis using its command syntax?

easy

A. SET key value

B. GET key value

C. STORE key value

D. PUT key value

Solution

1. Step 1: Recall Redis command for setting data

   Redis uses the command SET key value to store a value under a key.

2. Step 2: Verify other options

   GET retrieves data, STORE and PUT are not valid Redis commands.

3. Final Answer:

   SET key value -> Option A

4. Quick Check:

   Redis set command = SET key value [OK]

Hint: Remember Redis uses SET to store key-value [OK]

Common Mistakes:

• Confusing GET with SET
• Using non-existent commands like STORE or PUT
• Mixing Redis commands with SQL syntax

3. Consider this simplified Redis usage in code:

cache.set('user:1', 'Alice')
name = cache.get('user:1')
print(name)

What will be printed?

medium

A. None

B. 'Alice'

C. user:1

D. Error

Solution

1. Step 1: Analyze the cache set operation

   The key 'user:1' is set with the value 'Alice' in the cache.

2. Step 2: Analyze the cache get operation

   Retrieving 'user:1' returns the stored value 'Alice', which is then printed.

3. Final Answer:

   'Alice' -> Option B

4. Quick Check:

   Set then get returns stored value = 'Alice' [OK]

Hint: Get returns value set by set command [OK]

Common Mistakes:

• Expecting key name instead of value
• Assuming None if not careful with key format
• Thinking it causes an error without context

4. A developer tries to use Memcached but notices cache misses even after setting keys. Which of these is a likely cause?

medium

A. The database is down, so cache cannot be used

B. Memcached does not support key-value storage

C. The cache keys are expiring too quickly due to a low TTL setting

D. Memcached requires SQL queries to retrieve data

Solution

1. Step 1: Understand Memcached key expiration

   Memcached keys have a TTL (time to live). If TTL is too low, keys expire quickly causing cache misses.

2. Step 2: Eliminate incorrect options

   Memcached supports key-value storage, does not depend on database status, and does not use SQL queries.

3. Final Answer:

   The cache keys are expiring too quickly due to a low TTL setting -> Option C

4. Quick Check:

   Low TTL causes cache misses = A [OK]

Hint: Check TTL settings if cache misses occur [OK]

Common Mistakes:

• Thinking Memcached lacks key-value support
• Assuming database status affects cache directly
• Confusing Memcached with SQL databases

5. You are designing a high-traffic web app that uses Redis as a distributed cache. To ensure data consistency when multiple servers update the same cache key, which approach is best?

hard

A. Use Memcached instead because it handles concurrency automatically

B. Allow all servers to update cache keys without coordination

C. Disable caching for keys that are frequently updated

D. Use Redis transactions or Lua scripts to perform atomic updates

Solution

1. Step 1: Identify concurrency challenges in distributed cache

   When multiple servers update the same key, race conditions can cause inconsistent data.

2. Step 2: Choose atomic update methods in Redis

   Redis supports transactions and Lua scripts to ensure updates happen atomically, preventing conflicts.

3. Step 3: Evaluate other options

   Allowing uncoordinated updates risks inconsistency; disabling caching reduces performance; Memcached lacks built-in atomic scripting.

4. Final Answer:

   Use Redis transactions or Lua scripts to perform atomic updates -> Option D

5. Quick Check:

   Atomic updates ensure consistency = D [OK]

Hint: Use Redis atomic commands for safe concurrent updates [OK]

Common Mistakes:

• Ignoring concurrency causing stale data
• Thinking Memcached handles concurrency automatically
• Disabling cache unnecessarily reduces speed