Cache-aside pattern in HLD - System Design Exercise

Design: Cache-aside Pattern Implementation

Design focuses on the cache-aside pattern for read-heavy workloads with a relational database and an in-memory cache. Write-heavy scenarios and distributed cache synchronization are out of scope.

Functional Requirements

FR1: The system should efficiently serve data requests by using a cache to reduce database load.

FR2: When data is requested, the system should first check the cache.

FR3: If data is not in the cache (cache miss), the system should fetch it from the database and update the cache.

FR4: The cache should be updated or invalidated when data changes in the database to maintain consistency.

FR5: The system should handle concurrent requests safely without stale or inconsistent data.

FR6: The cache should improve read performance significantly compared to direct database access.

Non-Functional Requirements

NFR1: The system should handle up to 10,000 concurrent read requests with p99 latency under 100ms.

NFR2: Cache consistency should be eventual but with minimal stale data window (under 5 seconds).

NFR3: Availability target is 99.9% uptime.

NFR4: Cache size is limited and should evict least recently used data when full.

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

Client application or service

In-memory cache (e.g., Redis, Memcached)

Primary database (e.g., PostgreSQL, MySQL)

Cache management logic in the application layer

Design Patterns

Cache-aside (Lazy loading)

Cache invalidation strategies

Cache eviction policies (LRU, TTL)

Read-through and write-through caching (for comparison)

Reference Architecture

Client
  |
  v
Application Layer
  |
  |---> Cache (Redis/Memcached)
  |       |
  |       v
  |    Cache Miss?
  |       |
  |       v
  |    Database (PostgreSQL/MySQL)
  |
  v
Response to Client

Components

Client

Any frontend or service

Sends data requests to the application

Application Layer

Backend service (Node.js, Java, Python, etc.)

Implements cache-aside logic: checks cache, fetches from DB on miss, updates cache

Cache

Redis or Memcached

Stores frequently accessed data in memory for fast retrieval

Database

Relational DB like PostgreSQL or MySQL

Stores persistent data; source of truth

Request Flow

1. Client sends a data request to the application.

2. Application checks the cache for requested data.

3. If data is found in cache (cache hit), return data to client immediately.

4. If data is not found (cache miss), application queries the database.

5. Database returns data to application.

6. Application stores the retrieved data in cache for future requests.

7. Application returns data to client.

8. When data updates occur, application invalidates or updates the cache to keep data consistent.

Database Schema

Entities depend on application domain; cache stores serialized data keyed by unique identifiers matching database primary keys. Relationships remain in the database; cache holds denormalized or frequently accessed data subsets.

Scaling Discussion

Bottlenecks

Cache size limits causing frequent evictions and cache misses.

High write volume causing cache invalidation overhead.

Database becoming a bottleneck on cache misses under heavy load.

Cache consistency delays leading to stale data served to clients.

Solutions

Implement cache sharding or distributed cache clusters to increase capacity.

Use asynchronous cache invalidation or write-back strategies to reduce overhead.

Scale database vertically or horizontally (read replicas) to handle increased load.

Use TTL (time-to-live) and event-driven cache updates to minimize stale data window.

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying constraints, 20 minutes designing the architecture and data flow, 10 minutes discussing scaling and trade-offs, and 5 minutes summarizing.

Explain the cache-aside pattern clearly with real-world analogy (like checking a fridge before going to the store).

Discuss cache hit and miss scenarios and how they affect performance.

Highlight cache invalidation importance to avoid stale data.

Mention trade-offs between consistency and performance.

Address scaling challenges and practical solutions.