Distributed caching (Redis, Memcached) in HLD - System Design Exercise

Design: Distributed Caching System

Design the caching layer architecture using Redis or Memcached for a web application backend. Out of scope: database design, client application logic, and detailed security mechanisms.

Functional Requirements

FR1: Cache frequently accessed data to reduce database load

FR2: Support high read throughput with low latency (p99 < 10ms)

FR3: Handle 100,000 concurrent cache requests

FR4: Provide cache consistency with eventual consistency model

FR5: Support cache invalidation and expiration policies

FR6: Ensure high availability with 99.9% uptime

FR7: Allow horizontal scaling of cache nodes

Non-Functional Requirements

NFR1: Cache size limited to 100GB per node

NFR2: Network latency between cache nodes and clients < 5ms

NFR3: Data loss acceptable for short periods (cache is a performance layer)

NFR4: Cache miss penalty should not exceed 50ms additional latency

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

❓ Question 7

Key Components

Cache nodes (Redis or Memcached instances)

Cache clients integrated with application servers

Cache coordinator or proxy (optional)

Cache invalidation mechanism

Monitoring and alerting system

Load balancer or DNS for cache node discovery

Design Patterns

Cache-aside pattern

Write-through and write-back caching

Consistent hashing for cache sharding

Replication for high availability

Cache warming and preloading

Cache invalidation strategies

Reference Architecture

Application Servers

↓

Cache Clients

↓

Cache Node 1

↓

Persistent DB

Components

Cache Nodes

Redis or Memcached

Store cached data in memory for fast access with support for sharding and replication

Cache Clients

Language-specific Redis/Memcached clients

Interface between application servers and cache nodes to read/write cached data

Application Servers

Any backend framework

Serve user requests, query cache first, fallback to database on cache miss

Persistent Database

Relational or NoSQL DB

Source of truth for data, updated on writes

Monitoring & Alerting

Prometheus, Grafana, or similar

Track cache hit rates, latency, node health, and trigger alerts

Request Flow

1. Client sends request to Application Server

2. Application Server queries Cache Client for data using cache key

3. Cache Client checks Distributed Cache Cluster

4. If cache hit, data returned immediately to Application Server

5. If cache miss, Application Server queries Persistent Database

6. Application Server updates cache with fresh data (cache-aside pattern)

7. Application Server returns data to Client

8. Cache nodes replicate data asynchronously for availability

9. Cache entries expire or are evicted based on TTL or LRU policies

Database Schema

Not applicable - caching layer stores key-value pairs with keys representing data identifiers and values as serialized data objects. No fixed schema required.

Scaling Discussion

Bottlenecks

Single cache node memory limits causing cache misses

Network bandwidth saturation between application and cache nodes

Cache node failure causing data unavailability

Cache stampede on popular keys causing database overload

Inconsistent cache data due to delayed invalidation

Solutions

Use consistent hashing to shard cache keys across multiple nodes to scale memory horizontally

Deploy cache nodes in the same data center or availability zone to reduce network latency

Implement replication and failover mechanisms for cache nodes to ensure high availability

Use request coalescing or locking to prevent cache stampede on popular keys

Adopt cache invalidation strategies like write-through or event-driven invalidation to maintain freshness

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing and answering questions.

Explain cache-aside pattern and why it fits many use cases

Discuss trade-offs between Redis and Memcached (persistence, data structures)

Highlight importance of cache invalidation and consistency models

Describe how consistent hashing helps scale cache horizontally

Mention monitoring and alerting to maintain cache health

Address failure scenarios and recovery strategies

Practice

1. What is the primary purpose of using a distributed cache like Redis or Memcached in a system?

easy

A. To reduce database load by storing frequently accessed data in memory

B. To permanently store all application data for backup

C. To replace the database entirely for all data storage needs

D. To increase the size of the database by adding more servers

Solution

1. Step 1: Understand the role of distributed cache

   Distributed caches store frequently accessed data in fast memory to reduce the number of database queries.

2. Step 2: Identify the main benefit

   This reduces the load on the database and speeds up data retrieval for repeated requests.

3. Final Answer:

   To reduce database load by storing frequently accessed data in memory -> Option A

4. Quick Check:

   Distributed cache reduces DB load = C [OK]

Hint: Cache stores frequent data to reduce DB hits [OK]

Common Mistakes:

• Thinking cache permanently stores all data
• Confusing cache with database backup
• Assuming cache replaces the database

2. Which of the following is the correct way to set a key-value pair in Redis using its command syntax?

easy

A. SET key value

B. GET key value

C. STORE key value

D. PUT key value

Solution

1. Step 1: Recall Redis command for setting data

   Redis uses the command SET key value to store a value under a key.

2. Step 2: Verify other options

   GET retrieves data, STORE and PUT are not valid Redis commands.

3. Final Answer:

   SET key value -> Option A

4. Quick Check:

   Redis set command = SET key value [OK]

Hint: Remember Redis uses SET to store key-value [OK]

Common Mistakes:

• Confusing GET with SET
• Using non-existent commands like STORE or PUT
• Mixing Redis commands with SQL syntax

3. Consider this simplified Redis usage in code:

cache.set('user:1', 'Alice')
name = cache.get('user:1')
print(name)

What will be printed?

medium

A. None

B. 'Alice'

C. user:1

D. Error

Solution

1. Step 1: Analyze the cache set operation

   The key 'user:1' is set with the value 'Alice' in the cache.

2. Step 2: Analyze the cache get operation

   Retrieving 'user:1' returns the stored value 'Alice', which is then printed.

3. Final Answer:

   'Alice' -> Option B

4. Quick Check:

   Set then get returns stored value = 'Alice' [OK]

Hint: Get returns value set by set command [OK]

Common Mistakes:

• Expecting key name instead of value
• Assuming None if not careful with key format
• Thinking it causes an error without context

4. A developer tries to use Memcached but notices cache misses even after setting keys. Which of these is a likely cause?

medium

A. The database is down, so cache cannot be used

B. Memcached does not support key-value storage

C. The cache keys are expiring too quickly due to a low TTL setting

D. Memcached requires SQL queries to retrieve data

Solution

1. Step 1: Understand Memcached key expiration

   Memcached keys have a TTL (time to live). If TTL is too low, keys expire quickly causing cache misses.

2. Step 2: Eliminate incorrect options

   Memcached supports key-value storage, does not depend on database status, and does not use SQL queries.

3. Final Answer:

   The cache keys are expiring too quickly due to a low TTL setting -> Option C

4. Quick Check:

   Low TTL causes cache misses = A [OK]

Hint: Check TTL settings if cache misses occur [OK]

Common Mistakes:

• Thinking Memcached lacks key-value support
• Assuming database status affects cache directly
• Confusing Memcached with SQL databases

5. You are designing a high-traffic web app that uses Redis as a distributed cache. To ensure data consistency when multiple servers update the same cache key, which approach is best?

hard

A. Use Memcached instead because it handles concurrency automatically

B. Allow all servers to update cache keys without coordination

C. Disable caching for keys that are frequently updated

D. Use Redis transactions or Lua scripts to perform atomic updates

Solution

1. Step 1: Identify concurrency challenges in distributed cache

   When multiple servers update the same key, race conditions can cause inconsistent data.

2. Step 2: Choose atomic update methods in Redis

   Redis supports transactions and Lua scripts to ensure updates happen atomically, preventing conflicts.

3. Step 3: Evaluate other options

   Allowing uncoordinated updates risks inconsistency; disabling caching reduces performance; Memcached lacks built-in atomic scripting.

4. Final Answer:

   Use Redis transactions or Lua scripts to perform atomic updates -> Option D

5. Quick Check:

   Atomic updates ensure consistency = D [OK]

Hint: Use Redis atomic commands for safe concurrent updates [OK]

Common Mistakes:

• Ignoring concurrency causing stale data
• Thinking Memcached handles concurrency automatically
• Disabling cache unnecessarily reduces speed