Bird
Raised Fist0

Why Caching Exists and the Real Cost of Not Caching

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🎯
Why Caching Exists and the Real Cost of Not Caching
45 min · senior
senior

You are designing a backend system for a startup with 50 million daily active users (DAU). The system must serve user profile data and frequently accessed metadata with a P99 latency under 200ms. Data must be encrypted at rest and in transit. The system should support offline delivery of updates to clients. The backend database is a highly available distributed SQL store. The business requires that the system handle peak read QPS of 1 million requests per second with minimal backend load.

Constraints
  • Data must be encrypted at rest and in transit
  • P99 latency must be under 200ms for read requests
  • Support offline delivery of updates to clients
  • Peak read QPS of 1 million requests per second
  • Backend database is a distributed SQL store with limited read throughput
  • Cache consistency must be maintained within 5 seconds of data changes
📋
Guided Steps
1/5
Step 1
Identify the need for caching and its impact on backend load
Explain why caching is necessary at this scale and what happens if caching is not used.
💡 Hint
Consider the read QPS and backend database throughput limits.
Key Decision
Use Redis Cluster as a distributed in-memory cache to serve 1M QPS reads, offloading the distributed SQL store which cannot handle this read volume.
Interviewer Expectation
Mid-level: Names Redis as cache. Senior: Explains Redis Cluster sharding to handle 1M QPS and how direct DB reads would saturate backend causing high latency and failures.
⚠️Common Wrong Answer
Relying solely on the distributed SQL database for reads - leads to backend overload, increased latency, and potential outages at 1M QPS.
Step 2
Design cache consistency and update propagation
How will you keep the cache consistent with the backend database within 5 seconds of data changes?
💡 Hint
Think about change data capture and event-driven cache invalidation.
Key Decision
Use Debezium CDC on the distributed SQL database to stream change events into Kafka, then have cache update workers consume Kafka topics to update Redis within seconds.
Interviewer Expectation
Mid-level: Mentions cache invalidation on writes. Senior: Details Debezium CDC streaming changes to Kafka, explains why polling DB for changes is inefficient and causes stale cache.
⚠️Common Wrong Answer
Polling the database every few seconds for changes - causes high DB load and inconsistent cache state leading to stale reads.
Step 3
Support offline delivery and client synchronization
How will clients sync updates after being offline and ensure they have the latest data?
💡 Hint
Consider versioning or timestamp-based incremental sync mechanisms.
Key Decision
Implement a versioned data model with Redis Streams to store recent changes; clients request updates since last known version to sync incrementally.
Interviewer Expectation
Mid-level: Suggests full data reload after offline. Senior: Explains Redis Streams for incremental sync, reducing bandwidth and latency, and how versioning prevents missed updates.
⚠️Common Wrong Answer
Clients always reload full data after offline - causes high bandwidth, latency spikes, and poor user experience.
Step 4
Design API endpoints to support caching and offline delivery
Sketch 3 core API endpoints that enable clients to read cached data, sync updates, and acknowledge receipt.
💡 Hint
Think about endpoints for fetching data, incremental sync, and update acknowledgments.
Key Decision
Define GET /profile/{user_id} to fetch cached profile data from Redis; GET /profile/updates?since=version for incremental updates from Redis Streams; POST /profile/ack to acknowledge update receipt.
Interviewer Expectation
Mid-level: Names basic GET and POST endpoints. Senior: Explains how GET hits Redis cache, incremental sync reduces load, and acknowledgments enable reliable offline delivery.
⚠️Common Wrong Answer
Single GET endpoint without incremental sync or acknowledgments - leads to inefficient data transfer and no offline support.
Step 5
Handle encryption and security in caching layer
How will you ensure data encryption at rest and in transit in the caching layer?
💡 Hint
Consider Redis encryption features and TLS for client-server communication.
Key Decision
Use Redis Enterprise with TLS enabled for in-transit encryption and enable Redis data encryption at rest using disk encryption and encrypted snapshots.
Interviewer Expectation
Mid-level: Mentions TLS for Redis. Senior: Details Redis Enterprise encryption at rest, key management integration, and compliance with security policies.
⚠️Common Wrong Answer
Ignoring encryption in cache layer - exposes sensitive data and violates compliance requirements.
🔌
API Design Challenge
Sketch 3 core API endpoints to support caching and offline delivery in this system.
Hints
  • How does the client request fresh data and fallback to cache?
  • How does the client sync updates after being offline?
  • How does the client acknowledge receipt of updates?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Use of Redis Cluster to handle 1M QPS read load
  • Change Data Capture with Debezium and Kafka for cache consistency
  • Incremental sync mechanism using Redis Streams or equivalent
  • API design supporting cached reads, incremental sync, and acknowledgments
Good to Have - 30%
  • Encryption at rest and in transit in caching layer
  • Explanation of cache invalidation latency and its impact on P99 latency
  • Discussion of trade-offs between cache TTL and consistency guarantees
Bonus - 10%
  • Use of Redis Enterprise features for security compliance
  • Handling cache failures gracefully with fallback to DB reads
  • Metrics and monitoring strategy for cache hit ratio and latency
Anti-patterns to Avoid
Relying solely on the backend database for all reads - causes backend overload and high latency at 1M QPS, leading to system outages and poor user experience
Polling the database for changes instead of CDC - leads to stale cache and increased DB load, resulting in inconsistent data served to clients
Clients always reloading full data after offline - causes bandwidth spikes, latency spikes, and poor user experience due to inefficient data transfer
Ignoring encryption in caching layer - exposes sensitive data and violates compliance requirements, risking data breaches
Single API endpoint without incremental sync or acknowledgments - leads to inefficient data transfer and no offline support, increasing latency and bandwidth usage
Using a single Redis node instead of cluster - causes bottlenecks and single point of failure, reducing system scalability and availability
✅
Model Answer
▼ Reveal

At 50M DAU and 1M QPS, caching with Redis Cluster is essential to offload the backend DB and meet P99 latency under 200ms. Change Data Capture via Debezium streams updates into Kafka, enabling cache workers to update Redis within 5 seconds, maintaining consistency. Redis Streams support incremental client sync after offline periods, reducing bandwidth and latency. API endpoints include GET /profile/{user_id} for cached reads, GET /profile/updates?since=version for incremental updates, and POST /profile/ack for acknowledgments. Redis Enterprise with TLS and encryption at rest ensures data security.

Key Decisions
  • Use Redis Cluster to horizontally scale caching and handle 1M QPS read load
  • Implement Debezium CDC on distributed SQL database to stream change events into Kafka for near real-time cache updates
  • Use Kafka as a durable event bus to decouple DB changes from cache update workers, ensuring reliability and scalability
  • Leverage Redis Streams to store incremental updates enabling efficient offline client synchronization
  • Adopt a versioned data model to track changes and enable incremental synchronization, preventing missed updates
  • Design API endpoints: GET /profile/{user_id} for cached reads, GET /profile/updates?since=version for incremental updates, and POST /profile/ack for update acknowledgments
  • Enable Redis Enterprise features including TLS for in-transit encryption and disk encryption for data at rest to meet security requirements
  • Maintain cache consistency within 5 seconds by having cache update workers consume Kafka topics promptly
  • Redis Cluster for distributed in-memory caching to handle 1M QPS read load
  • Debezium CDC streaming changes from distributed SQL database into Kafka for near real-time cache updates
  • Kafka as durable event bus to decouple DB changes and cache update workers
  • Redis Streams to store incremental updates enabling efficient offline client sync
  • Versioned data model to track changes and enable incremental synchronization
  • API endpoints: GET /profile/{user_id}, GET /profile/updates?since=version, POST /profile/ack for caching and offline delivery
  • Redis Enterprise with TLS and encryption at rest to meet security constraints
  • Cache update workers consuming Kafka topics to maintain cache consistency within 5 seconds

Practice

(1/5)
1. A distributed cache system needs to evenly distribute keys across servers and minimize data reshuffling when nodes are added or removed. Which component best handles this requirement?
easy
A. A simple modulo-based hash function assigning keys to servers by key % server_count
B. A centralized directory service mapping keys to servers explicitly
C. A consistent hashing ring with virtual nodes to balance load and reduce remapping
D. A random key assignment with periodic rebalancing of all keys

Solution

  1. Step 1: Understand key distribution and remapping

    Modulo-based hashing (A simple modulo-based hash function assigning keys to servers by key % server_count) causes massive remapping when server count changes, violating minimal reshuffling.

  2. Step 2: Evaluate consistent hashing with virtual nodes

    Consistent hashing with virtual nodes (A consistent hashing ring with virtual nodes to balance load and reduce remapping) evenly distributes keys and minimizes remapping on node changes.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Virtual nodes in consistent hashing minimize key remapping on node changes [check]
Hint: Virtual nodes smooth load, minimize reshuffle [check]
Common Mistakes:
2. In a messaging system, at step 4 the Delivery Worker checks the message queue for undelivered messages. What happens if the message's delivery TTL has expired at this step?
easy
A. The Delivery Worker marks the message as expired and removes it from the queue
B. The Delivery Worker discards the message without retrying or marking it
C. The Delivery Worker moves the message to a dead-letter queue but keeps it in the main queue
D. The Delivery Worker retries sending the message indefinitely until success

Solution

  1. Step 1: Understand Delivery Worker behavior at Step 4

    It checks message TTL to decide if delivery attempts should continue.

  2. Step 2: TTL expiration handling

    Expired messages are discarded without retry or marking to avoid resource waste; indefinite retries (A) cause resource exhaustion; marking expired (B) is not standard; dead-letter queue (C) is for failed, not expired messages.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Expired messages are discarded to maintain throughput and latency balance [check]
Hint: Expired messages get discarded immediately, no retries [check]
Common Mistakes:
3. A distributed caching system needs to assign keys to cache nodes such that minimal remapping occurs when nodes join or leave. Which component best handles this requirement?
easy
A. Use Rendezvous Hashing to assign keys to nodes with highest hash score
B. Use a simple modulo-based hash function on node count
C. Use a centralized directory service mapping keys to nodes
D. Use consistent hashing with virtual nodes and a hash ring

Solution

  1. Step 1: Understand minimal remapping requirement

    Modulo-based hashing (Use a simple modulo-based hash function on node count) causes massive remapping when node count changes. Centralized directory (Use a centralized directory service mapping keys to nodes) adds single point of failure and latency. Consistent hashing with virtual nodes (Use consistent hashing with virtual nodes and a hash ring) reduces remapping but is more complex and less direct than Rendezvous Hashing.

  2. Step 2: Identify Rendezvous Hashing benefits

    Rendezvous Hashing assigns keys to nodes by computing scores and picking the highest, minimizing remapping and avoiding ring maintenance.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Rendezvous Hashing picks highest score node -> minimal remapping [check]
Hint: Highest hash score wins key assignment [check]
Common Mistakes:
4. At 10x transaction volume, which component is most likely to become the first bottleneck in the payment system, and why?
hard
A. The client SDK, because it cannot handle increased retry logic under load
B. The Idempotency Store, because it is a centralized stateful component with limited horizontal scaling
C. The Settlement Service, due to batch processing delays and database write contention
D. The load balancer, as it cannot distribute traffic evenly at very high request rates

Solution

  1. Step 1: Identify stateful bottlenecks at scale

    The Idempotency Store holds global state and is hard to shard, limiting horizontal scaling.

  2. Step 2: Evaluate other components

    Settlement Service can be scaled with partitioned batches; client SDK is client-side and less critical; load balancer scales horizontally with modern tech.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Centralized idempotency store -> scaling bottleneck at 10x load [check]
Hint: Centralized stateful store limits scaling first [check]
Common Mistakes:
5. At 10x scale, which component is most likely to become the bottleneck in a WebSocket-based chat system, and why?
hard
A. The chat servers, because WebSocket connections are stateful and bound to server memory, making horizontal scaling and connection migration difficult.
B. The database, because it cannot handle the increased write volume from message persistence.
C. The load balancer, because it cannot efficiently distribute WebSocket connections due to their stateful nature.
D. The client browsers, because they cannot maintain more than a few WebSocket connections simultaneously.

Solution

  1. Step 1: Identify stateful components in WebSocket architecture

    WebSocket connections are stateful and maintained in server memory, binding clients to specific chat servers.

  2. Step 2: Analyze scaling challenges

    At 10x scale, chat servers face difficulty resharding connections or migrating them, limiting horizontal scaling. Databases and load balancers can be scaled more easily. Client browser limits are less impactful at system scale.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    WebSocket stateful connections -> chat servers bottleneck at scale [check]
Hint: Stateful WebSocket connections bind clients to servers -> chat servers bottleneck [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard