Bird
Raised Fist0

Durability - How Systems Guarantee Data is Never Lost

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/5
Write Coordinator

Receives client write request

Write Coordinator accepts data and prepares to persist

Protocol:gRPC Write Request
BeforeClient data exists only on client device
AfterWrite Coordinator holds data ready to publish to Kafka
Kafka Cluster

Write Coordinator publishes data

Data is appended to Kafka topic partition with replication

Protocol:Kafka Produce
BeforeData held only in Write Coordinator memory
AfterData durably committed in Kafka with replication factor
Replication Service

Consumes data from Kafka

Reads committed data and writes to Cassandra nodes

Protocol:Kafka Consume + CQL Write
BeforeData durable in Kafka but not yet in Cassandra
AfterReplication Service has written data to Cassandra nodes
Storage Nodes (Cassandra)

Persist data

Data is stored on multiple nodes with consistency guarantees

Protocol:CQL Write
BeforeData not yet stored in Cassandra cluster
AfterData durably stored and replicated in Cassandra cluster
Read API Layer

Serves read request

Reads data from Cassandra and returns to client

Protocol:CQL Read
BeforeData stored in Cassandra but not yet read
AfterClient receives durable, consistent data from storage
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────────┐
│ Write Coordinator │
└─────────┬─────────┘
          │ gRPC Write Request
          ▼
┌───────────────────┐
│  Kafka Cluster    │
│ (Durable Queue)   │
└─────────┬─────────┘
          │ Kafka Replication
          ▼
┌────────────────────────┐
│ Replication Service    │
│ (Copies data to Cass)  │
└─────────┬──────────────┘
          │ Cassandra Write (CQL)
          ▼
┌────────────────────────┐
│ Storage Nodes (Cass)   │
│ (Persistent Storage)   │
└─────────┬──────────────┘
          │ Read Request (CQL)
          ▼
┌───────────────────┐
│  Read API Layer   │
│ (Serves client    │
│  read queries)    │
└───────────────────┘
Design Rationale

This architecture ensures durability by first writing data to a Kafka cluster, which acts as a durable, replicated commit log. The Write Coordinator accepts client writes and publishes them to Kafka, guaranteeing data persistence even if downstream components fail. The Replication Service consumes Kafka data and writes it to Cassandra Storage Nodes, which provide long-term distributed storage with replication and consistency. Finally, the Read API Layer serves client read requests from Cassandra, ensuring clients only see data that is durably stored. This multi-stage pipeline prevents data loss by combining Kafka's durable log with Cassandra's replicated storage, ensuring data is never lost even during failures.

Primary Focus

✓

Practice

(1/5)
1. A location-based service must efficiently index millions of businesses worldwide to support fast nearby searches within a 5km radius. Which component is best suited to handle spatial indexing for this use case?
easy
A. Use a relational database with latitude and longitude columns and perform range queries with indexes
B. Use a Geohash-based spatial index to bucket locations into hierarchical grids
C. Use a simple key-value store with business IDs as keys and coordinates as values
D. Use a full-text search engine like Elasticsearch without spatial extensions

Solution

  1. Step 1: Identify spatial indexing needs

    Relational DBs with lat/lon indexes (Use a relational database with latitude and longitude columns and perform range queries with indexes) do not scale well for large radius queries and millions of points due to expensive range scans.

  2. Step 2: Evaluate Geohash suitability

    Geohash (Use a Geohash-based spatial index to bucket locations into hierarchical grids) encodes locations into hierarchical grids enabling efficient prefix queries for nearby searches, making it ideal for large-scale spatial indexing.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Geohash buckets locations into grids for fast proximity queries [check]
Hint: Geohash buckets locations hierarchically for fast spatial queries [OK]
Common Mistakes:
2. In a system trace, at Step 3 the Storage Estimator component calculates daily storage needs. If the average object size is 2MB and the system expects 500K new objects daily, what is the storage estimate at Step 3?
easy
A. Cannot estimate without knowing retention period
B. 1TB per day, calculated as 500K objects x 2MB each
C. 2TB per day, assuming 4MB average object size instead
D. 500GB per day, assuming 1MB average object size instead

Solution

  1. Step 1: Calculate daily storage

    Multiply 500,000 objects by 2MB each = 1,000,000 MB = 1TB per day.

  2. Step 2: Eliminate other options

    500GB per day, assuming 1MB average object size instead uses wrong object size; 2TB per day, assuming 4MB average object size instead doubles size incorrectly; Cannot estimate without knowing retention period ignores that retention affects total storage, not daily estimate.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    500K x 2MB = 1TB daily storage [check]
Hint: Daily storage = object count x average size [check]
Common Mistakes:
3. In a CQRS system, a write request updates the primary database at Step 3. At Step 5, the read replica receives an update event. What is the state of the read replica at Step 6 immediately after receiving the event but before applying it?
easy
A. The read replica has already applied the update and serves the new data
B. The read replica rejects all reads until the update event is fully processed
C. The read replica switches to a read-only mode and queues incoming reads
D. The read replica still serves stale data until the update event is applied

Solution

  1. Step 1: Trace event arrival at read replica

    Upon receiving the update event, the read replica has not yet applied it, so data is stale.

  2. Step 2: Understand read replica behavior

    Read replicas serve stale data until the event is processed; they do not reject or queue reads.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Read replicas serve stale data until update applied [check]
Hint: Read replicas lag briefly, serve stale data until updated [check]
Common Mistakes:
4. If the global BGP route monitoring system is disabled during a misconfiguration event, what is the most likely failure observed in the network recovery process?
medium
A. Routers automatically revert to default static routes, causing traffic blackholing
B. Network devices silently drop BGP updates, leading to partial route visibility
C. Operators receive delayed alerts, causing prolonged outage and inconsistent routing states
D. Clients experience immediate failover to backup ISPs without disruption

Solution

  1. Step 1: Role of BGP monitoring

    BGP monitoring detects anomalies and route changes; without it, routers may silently drop updates due to misconfigurations.

  2. Step 2: Eliminate other options

    Delayed alerts (A) affect operators but don't cause silent drops; routers don't revert to static routes automatically (B); immediate client failover (D) unlikely without proper routing.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Without monitoring, routers drop BGP updates silently causing partial visibility [check]
Hint: No monitoring -> silent BGP update drops, partial routes [check]
Common Mistakes:
5. At 10x traffic scale, which component in the cache warming architecture is most likely to become the bottleneck first, and why?
hard
A. The database, because it cannot handle the increased read load from the warming service
B. The network bandwidth, because warming floods the network with data transfers
C. The cache servers, because they run out of memory storing warmed keys
D. The cache warming coordinator, because it maintains global state and cannot shard effectively

Solution

  1. Step 1: Identify coordinator role and scaling limits

    The warming coordinator manages global warming state and task assignment, which is hard to shard.

  2. Step 2: Compare other components

    Database can be scaled with replicas; cache servers scale horizontally; network scales with capacity upgrades.

  3. Step 3: Conclude bottleneck

    Coordinator's centralized state causes it to fail first at 10x scale.

  4. Final Answer:

    Option D -> Option D

  5. Quick Check:

    Centralized coordinator state limits sharding, causing bottleneck [check]
Hint: Central coordinator state limits sharding, causing bottleneck [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard