Cache-aside pattern in HLD - Scalability & System Analysis

Scalability Analysis - Cache-aside pattern

Growth Table: Cache-aside Pattern Scaling

Users / Traffic	Cache Size	Cache Hit Rate	Database Load	Latency	Notes
100 users	Small (MBs)	Low to Medium	Low	Low	Cache warm-up ongoing, DB handles most reads
10,000 users	Medium (GBs)	Medium to High	Moderate	Moderate	Cache hit rate improves, DB load increases
1,000,000 users	Large (10s GBs)	High	Moderate to High	Low to Moderate	Cache critical to reduce DB load, eviction policies needed
100,000,000 users	Very Large (100s GBs+)	Very High	High	Low	Distributed cache clusters, sharding, and advanced eviction

First Bottleneck

The database becomes the first bottleneck as user traffic grows. Initially, the cache is cold and misses cause many DB reads. As traffic increases, the DB handles more queries, risking overload. Without an effective cache hit rate, DB CPU and I/O saturate first.

Scaling Solutions

• Cache Warm-up: Preload popular data to improve hit rate early.
• Horizontal Scaling: Add more cache nodes (distributed cache) to handle larger data and traffic.
• Eviction Policies: Use LRU or LFU to keep cache fresh and relevant.
• Database Read Replicas: Offload read queries to replicas to reduce primary DB load.
• Sharding: Partition data across multiple DB instances to scale writes and reads.
• Connection Pooling: Efficiently manage DB connections to avoid overload.
• Monitoring & Alerts: Track cache hit rates and DB load to react proactively.

Back-of-Envelope Cost Analysis

Assuming 1M users with 10 requests per second each:

• Total requests: 10 million QPS
• Cache hit rate: 80% -> 2 million DB queries per second
• Single DB instance handles ~10,000 QPS -> Need ~200 DB replicas
• Cache memory: 100 GB+ distributed across nodes
• Network bandwidth: 10M QPS * 1 KB/request ≈ 10 GB/s (distributed)

Scaling cache and DB horizontally is essential to handle this load.

Interview Tip

Start by explaining the cache-aside pattern basics: cache misses trigger DB reads, then data is cached. Discuss how cache hit rate affects DB load. Identify the DB as the first bottleneck at scale. Propose solutions like cache warm-up, horizontal scaling, and read replicas. Use numbers to justify your approach.

Self Check Question

Your database handles 1000 QPS. Traffic grows 10x to 10,000 QPS. What do you do first?

Answer: Implement or improve the cache-aside pattern to increase cache hit rate and reduce DB queries. This reduces DB load before scaling DB horizontally.

Key Result

The database is the first bottleneck as traffic grows; improving cache hit rate and scaling cache horizontally are key to reducing DB load and maintaining low latency.

Practice

1. What is the main purpose of the cache-aside pattern in system design?

easy

A. To load data into cache only when it is requested

B. To preload all data into cache before any request

C. To write data directly to cache without database interaction

D. To keep cache and database completely separate without synchronization

Solution

1. Step 1: Understand cache-aside pattern behavior

   The cache-aside pattern loads data into cache only when the application requests it and the data is not already cached.

2. Step 2: Compare options with this behavior

   To load data into cache only when it is requested correctly states this lazy loading approach. Other options describe preloading or unsynchronized caching, which are incorrect.

3. Final Answer:

   To load data into cache only when it is requested -> Option A

4. Quick Check:

   Cache-aside = lazy loading on demand [OK]

Hint: Cache-aside loads data only when requested [OK]

Common Mistakes:

• Thinking cache preloads all data
• Assuming cache updates happen before DB writes
• Believing cache and DB are unsynchronized

2. Which of the following is the correct sequence of operations in the cache-aside pattern when reading data?

easy

A. Read from database -> Update cache -> Return data

B. Read from cache -> If miss, read from database -> Update cache

C. Update cache -> Read from cache -> Read from database

D. Read from cache -> Return data without checking database

Solution

1. Step 1: Recall cache-aside read flow

   When reading, the system first checks the cache. If data is missing (cache miss), it reads from the database and then updates the cache.

2. Step 2: Match this flow to options

   Read from cache -> If miss, read from database -> Update cache matches this exact sequence. Other options either skip cache miss handling or update cache before reading.

3. Final Answer:

   Read from cache -> If miss, read from database -> Update cache -> Option B

4. Quick Check:

   Cache read -> DB read on miss -> Cache update [OK]

Hint: Cache read first, DB read only if cache miss [OK]

Common Mistakes:

• Updating cache before reading database
• Ignoring cache miss and reading only cache
• Reading database before checking cache

3. Consider this pseudocode using cache-aside pattern:

function getUser(id) {
  user = cache.get(id)
  if (user == null) {
    user = database.query(id)
    cache.set(id, user)
  }
  return user
}

What will getUser(5) return if the cache is empty and the database has user with id 5?

medium

A. null

B. Error due to cache miss

C. Empty object

D. User data from database

Solution

1. Step 1: Analyze cache miss handling

   Since cache is empty, cache.get(5) returns null, triggering database query.

2. Step 2: Database query and cache update

   The function queries the database for user id 5, finds the user data, updates the cache, and returns the user data.

3. Final Answer:

   User data from database -> Option D

4. Quick Check:

   Cache miss triggers DB read and cache update [OK]

Hint: Cache miss leads to DB read and cache update [OK]

Common Mistakes:

• Assuming cache miss returns null directly
• Expecting error on cache miss
• Ignoring cache update after DB read

4. A developer implemented cache-aside but notices stale data in cache after database updates. What is the likely mistake?

medium

A. Not updating cache after database write

B. Reading from cache before database

C. Preloading cache with all data

D. Using cache for write operations only

Solution

1. Step 1: Identify cause of stale cache

   Stale cache means cache has old data not reflecting recent database updates.

2. Step 2: Check cache update after DB write

   If cache is not updated or invalidated after database writes, stale data remains in cache.

3. Final Answer:

   Not updating cache after database write -> Option A

4. Quick Check:

   Cache must update after DB write to avoid stale data [OK]

Hint: Always update cache after DB write to avoid stale data [OK]

Common Mistakes:

• Ignoring cache invalidation after writes
• Assuming cache updates automatically
• Confusing read and write cache operations

5. You design a high-traffic product catalog service using cache-aside pattern. To handle frequent updates and reads, which strategy best maintains cache consistency and performance?

hard

A. Preload entire product catalog into cache at service start and never update cache

B. On product update, write to DB only and let cache expire naturally

C. On product update, write to DB then immediately update cache with new data

D. Write updates only to cache and asynchronously sync to DB later

Solution

1. Step 1: Understand cache consistency needs

   For frequent updates and reads, cache must reflect latest data quickly to avoid stale reads.

2. Step 2: Evaluate update strategies

   On product update, write to DB then immediately update cache with new data updates DB then cache immediately, ensuring cache consistency and fast reads. On product update, write to DB only and let cache expire naturally risks stale cache. Preload entire product catalog into cache at service start and never update cache ignores updates. Write updates only to cache and asynchronously sync to DB later risks data loss and inconsistency.

3. Final Answer:

   On product update, write to DB then immediately update cache with new data -> Option C

4. Quick Check:

   Update DB then cache for consistency and performance [OK]

Hint: Update DB then cache immediately for fresh data [OK]

Common Mistakes:

• Relying on cache expiration alone
• Preloading cache without updates
• Writing only to cache without DB sync