Bird
Raised Fist0

Why Caching Exists and the Real Cost of Not Caching

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Why Caching Exists and the Real Cost of Not Caching
Understanding caching's role in system performance and cost at scale
mediumCachingFreq: common
⚙️
Functional Requirements
  • Serve user requests with low latency (sub-100ms P99) for read-heavy workloads
  • Cache frequently accessed data to reduce load on primary data stores
  • Support cache invalidation and TTL to ensure data freshness
  • Handle cache misses gracefully by falling back to primary storage
  • Provide metrics on cache hit/miss ratio and latency
  • Support horizontal scaling of cache layer to handle peak QPS
📊
Non-Functional Requirements
  • Support 2 billion daily active users (DAU)
  • Handle peak write throughput of ~925K writes/sec
  • Achieve 99.99% availability for read requests
  • Maintain 100ms P99 latency for cache lookups
  • Ensure cache consistency within eventual consistency bounds
❓
Clarifying Questions to Ask
  • What is the read-to-write ratio for the workload?
  • What is the acceptable staleness or TTL for cached data?
  • Are there any specific data types or sizes to cache?
  • What is the expected cache hit ratio target?
  • Is the cache shared across multiple services or per service?
  • What is the failure tolerance for cache unavailability?
Out of Scope
✗ Design of primary data store (e.g., database internals)
✗ Authentication and authorization mechanisms
✗ Detailed network security and encryption
✗ Complex multi-region replication strategies
📌
Assumptions
  • 2 billion DAU
  • Each user generates 40 messages per day
  • Average message size is 100 bytes
  • Read-to-write ratio is 10:1
⚡
QPS Calculation
write qps2B × 40 / 86400 ≈ 925,926 writes/sec
peak write qpsAssuming 2× peak factor → 1,851,852 writes/sec
read qpswrite_qps × 10 = 9,259,260 reads/sec
💾
Storage Estimate
daily storage2B × 40 × 100 bytes = 8 TB/day
retention days7
total storage8 TB × 7 = 56 TB
with replication56 TB × 3 replicas = 168 TB
🖥️
Server Estimate
cache serversAssuming Redis instance handles 50K QPS → 9,259,260 / 50,000 ≈ 186 servers
db serversAssuming Cassandra node handles 20K writes/sec → 1,851,852 / 20,000 ≈ 93 nodes
🏗️
Architecture Diagram
Architecture Diagram
          +-------------+
          |   Clients   |
          +------+------+
                 |
                 v
          +------+-------+
          |  API Gateway |
          +------+-------+
                 |
        +--------+---------+
        |                  |
        v                  v
  +-----+-----+      +-----+-----+
  |  Redis    |      | Cassandra |
  |  Cache    |      |  Primary  |
  +-----+-----+      +-----+-----+
        |                  |
        +--------+---------+
                 |
          +------+-------+
          | Monitoring & |
          |  Metrics     |
          +--------------+
🧩
Components
API Gateway
Entry point for client requests, routes to cache or DB
Why thisNginx with Lua scripting supports high QPS routing and caching logic at edge
Why notAWS ALB lacks fine-grained caching control and custom routing logic
Redis Cache Cluster
In-memory cache layer to serve hot data with sub-millisecond latency
Why thisRedis supports TTL natively, offers high throughput, and supports clustering for horizontal scale
Why notMemcached lacks persistence and advanced eviction policies needed for data freshness
Cassandra Primary Store
Durable storage for all messages with high write throughput and eventual consistency
Why thisCassandra scales linearly for writes, supports multi-region replication, and handles 1.8M writes/sec
Why notPostgreSQL requires complex manual sharding and does not scale writes linearly at this scale
Monitoring & Metrics
Collects cache hit/miss ratios, latency, and system health metrics
Why thisPrometheus + Grafana provide real-time observability and alerting for SLA adherence
Why notCloudWatch lacks granularity and real-time alerting needed for rapid troubleshooting
Cache Invalidation Service
Manages TTL and explicit invalidations to maintain cache consistency
Why thisCustom microservice using Kafka for event-driven invalidation ensures eventual consistency
Why notCron-based invalidation causes stale data spikes and does not scale with write QPS
🔄
Data Flow
1
Client → API Gateway
Client sends read/write request
API Gateway routes based on request type
2
API Gateway → Redis Cache
For reads, query cache first
Sub-millisecond lookup reduces latency if hit
3
Redis Cache → API Gateway
Return cached data if hit
Avoids expensive DB read
4
API Gateway → Cassandra
On cache miss or writes, query/write primary DB
Writes always go to DB; reads fallback on miss
5
Cassandra → Cache Invalidation Service
Emit events for cache invalidation or update
Keeps cache consistent with DB writes
6
Cache Invalidation Service → Redis Cache
Invalidate or update cache entries
Prevents stale data serving
7
Monitoring & Metrics → Operations Team
Expose cache hit/miss and latency metrics
Enables SLA monitoring and troubleshooting

Practice

(1/5)
1. A system must enforce a maximum number of API calls per user per minute to prevent abuse. Which component in an API Gateway architecture is best suited to handle this requirement?
easy
A. The API Gateway applies rate limiting before routing requests to backend services
B. The backend microservices implement rate limiting individually to distribute load
C. A centralized database tracks request counts and enforces limits synchronously
D. Clients are responsible for self-throttling their request rates

Solution

  1. Step 1: Identify rate limiting responsibility

    The API Gateway is the entry point and can efficiently enforce rate limits before requests reach backend services, preventing overload.

  2. Step 2: Eliminate other options

    The backend microservices implement rate limiting individually to distribute load spreads rate limiting, risking inconsistent enforcement and backend overload. A centralized database tracks request counts and enforces limits synchronously adds latency and complexity by synchronous DB checks. Clients are responsible for self-throttling their request rates trusts clients, which is insecure.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    API Gateway enforces rate limits upfront to protect backend [check]
Hint: Rate limiting belongs at the gateway entry point [check]
Common Mistakes:
2. In a system using HyperLogLog to estimate daily unique users, at step 3 the Aggregator merges multiple HyperLogLog sketches from different servers. What is the state of the merged sketch immediately after this step?
easy
A. The merged sketch contains exact unique user IDs from all servers
B. The merged sketch is an approximate union of all input sketches, preserving cardinality estimation
C. The merged sketch resets and only contains data from the last server processed
D. The merged sketch counts duplicates multiple times, inflating the estimate

Solution

  1. Step 1: Understand HyperLogLog merge operation

    Merging HyperLogLog sketches combines their registers by taking the maximum per register, producing an approximate union.

  2. Step 2: Check what the merged sketch represents

    The merged sketch approximates the cardinality of the union of all input sets, not exact IDs or duplicates.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    HyperLogLog merge -> approximate union, not exact data [check]
Hint: Merge = max registers -> approximate union preserved [check]
Common Mistakes:
3. If the origin server becomes unreachable, what exactly breaks in a CDN using origin pull with TTL caching?
medium
A. Edge nodes continue serving stale cached content indefinitely without errors
B. Edge nodes serve stale content until TTL expires, then return errors for new requests
C. Edge nodes immediately return errors for all requests without cached content
D. Edge nodes switch to push mode automatically to fetch content from other origins

Solution

  1. Step 1: Understand origin pull with TTL caching failure

    When origin is unreachable, edge nodes serve cached content until TTL expires.

  2. Step 2: Identify behavior after TTL expiry

    After TTL expires, edge nodes cannot refresh content and must return errors for new requests.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Origin down + expired TTL -> edge nodes return errors [check]
Hint: Expired TTL + origin down -> errors returned [check]
Common Mistakes:
4. If the Kafka cluster used for buffering document update events becomes unavailable, what is the immediate failure mode in the full-text search indexing pipeline?
medium
A. The system automatically switches to a backup message queue with identical ordering guarantees
B. Document update ingestion blocks, causing the entire indexing pipeline to stall and new updates to be lost if not retried
C. The indexing workers continue processing stale events from local cache, causing outdated search results
D. The inverted index becomes corrupted due to missing update events

Solution

  1. Step 1: Understand Kafka unavailability impact

    Without Kafka, ingestion of new document updates cannot proceed, causing blocking or stalling.

  2. Step 2: Evaluate options

    Indexing workers do not have local caches of events (A) and cannot continue processing. Automatic failover to a backup queue with identical ordering (C) is a designed mitigation. Index corruption (D) does not happen immediately; missing events cause stale results but not corruption.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Failover to backup queue prevents pipeline stall [check]
Hint: Kafka outage stalls ingestion unless backup queue with ordering [check]
Common Mistakes:
5. For a social media feed requiring low-latency reads and flexible schema, which database choice is better and why?
medium
A. Use a relational database with normalized tables for consistency
B. Use a wide-column NoSQL store to denormalize and optimize read latency
C. Use a graph database to model user relationships for feed generation
D. Use an in-memory cache only to serve all feed requests

Solution

  1. Step 1: Analyze read latency and schema flexibility needs

    Social feeds require fast reads and flexible schema to handle diverse content types.

  2. Step 2: Compare database options

    Relational DBs have join overhead; graph DBs excel at relationships but not high-volume feed reads; in-memory cache alone lacks persistence.

  3. Step 3: Justify wide-column NoSQL choice

    Wide-column stores denormalize data for fast reads and support flexible schemas, fitting feed requirements.

  4. Final Answer:

    Option B -> Option B

  5. Quick Check:

    Wide-column NoSQL -> denormalized, low-latency reads [check]
Hint: Low-latency feed -> denormalized wide-column store [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard