Bird
Raised Fist0

Consistency Models - Strong, Eventual, Causal, Read-Your-Writes Explained

HLDmediumInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Consistency Models - Strong, Eventual, Causal, Read-Your-Writes Explained
Design and explain distributed system consistency models with scale and failure reasoning
mediumCore Distributed Systems ConceptsFreq: high
⚙️
Functional Requirements
  • Support Strong Consistency guarantees for critical operations requiring linearizability
  • Support Eventual Consistency for high availability and partition tolerance scenarios
  • Support Causal Consistency to preserve causality between related operations
  • Support Read-Your-Writes consistency to ensure clients see their own updates immediately
  • Allow clients to select or negotiate consistency level per operation or session
  • Provide mechanisms to detect and resolve conflicts in weaker consistency models
📊
Non-Functional Requirements
  • System must handle 2 billion daily active users (DAU)
  • Write throughput up to 925,000 writes per second at peak
  • Read latency P99 under 100 milliseconds for strong consistency reads
  • Availability SLA of 99.99% even under network partitions
  • Data durability with replication factor of 3 across geographically distributed data centers
❓
Clarifying Questions to Ask
  • What are the target use cases for each consistency model in this system?
  • Is the system geo-distributed or single-region?
  • What is the expected read-to-write ratio?
  • Are clients stateful or stateless with respect to consistency sessions?
  • What is the maximum acceptable staleness for eventual consistency?
  • How are conflicts detected and resolved in causal and eventual models?
Out of Scope
✗ Authentication and authorization mechanisms
✗ Detailed network protocol design (e.g., TCP vs QUIC)
✗ Specific UI/UX consistency guarantees
✗ Data compression and encryption at rest
📌
Assumptions
  • 2 billion DAU
  • 40 messages per user per day
  • Average message size 100 bytes
  • Replication factor 3
⚡
QPS Calculation
write qps2,000,000,000 × 40 / 86,400 ≈ 925,926 writes/sec
peak write qpsAssuming 2× peak factor → 1,851,852 writes/sec
read qpsAssuming 10 reads per write → 9,259,260 reads/sec
💾
Storage Estimate
daily storage2,000,000,000 × 40 × 100 bytes = 8,000,000,000,000 bytes = 8 TB/day
retention days30
total storage8 TB × 30 = 240 TB
with replication240 TB × 3 = 720 TB
🖥️
Server Estimate
write frontendAssuming 10K writes/sec per server → 1,851,852 / 10,000 ≈ 186 servers
storage nodesAssuming 5 TB usable per node → 720 TB / 5 TB = 144 nodes
🏗️
Architecture Diagram
Architecture Diagram
          +----------------+
          |   Client App   |
          +--------+-------+
                   |
                   v
          +--------+-------+
          | Write Frontend |
          +--------+-------+
                   |
                   v
          +--------+-------+
          |  Kafka Cluster |
          +--------+-------+
                   |
                   v
          +--------+-------+
          | Cassandra DB   |
          +--------+-------+
                   |
                   v
          +--------+-------+
          | Consistency    |
          | Coordinator    |
          +--------+-------+
                   |
                   v
          +--------+-------+
          | Read Frontend  |
          +----------------+
🧩
Components
Write Frontend
Receives client writes, validates consistency level, and publishes to Kafka
Why thisKafka provides durable, partitioned, ordered log with high throughput and replication
Why notRabbitMQ lacks Kafka's throughput and partitioning needed for 1.8M writes/sec
Kafka Cluster
Durably stores ordered write events, enabling asynchronous replication and replay
Why thisKafka's partitioned log supports ordered writes and replay for causal and eventual consistency
Why notRedis Streams lacks Kafka's durability and partition scalability at this scale
Cassandra DB
Stores data with tunable consistency per query, supports multi-region replication
Why thisCassandra offers configurable consistency levels (QUORUM, ONE) and linear scalability
Why notPostgreSQL requires manual sharding and does not scale linearly to 925K writes/sec
Consistency Coordinator
Implements consistency protocols (strong, causal, read-your-writes) and conflict resolution
Why thisCustom service managing vector clocks and session tokens to enforce causal and read-your-writes
Why notRelying solely on DB consistency levels cannot enforce causal or read-your-writes semantics
Read Frontend
Serves client reads with requested consistency level, caching where applicable
Why thisMemcached caches hot data with TTL to reduce read latency under 100ms P99
Why notDirect DB reads cause high latency and load spikes, failing 100ms P99 SLA
🔄
Data Flow
1
Client App → Write Frontend
Client sends write with consistency level metadata
Consistency level guides processing path (strong, causal, eventual)
2
Write Frontend → Kafka Cluster
Write event published to Kafka partition based on key
Kafka ensures ordered, durable log for downstream processing
3
Kafka Cluster → Consistency Coordinator
Coordinator consumes events, applies consistency protocol logic
Vector clocks or session tokens updated for causal and read-your-writes
4
Consistency Coordinator → Cassandra DB
Writes data with appropriate consistency level (QUORUM for strong)
Cassandra replicates data across nodes and regions
5
Client App → Read Frontend
Client issues read with requested consistency level
Read Frontend serves from cache or DB ensuring consistency guarantees
6
Read Frontend → Cassandra DB
Reads data with consistency level (ONE for eventual, QUORUM for strong)
Latency optimized with caching and consistency trade-offs
7
Read Frontend → Client App
Returns data with consistency metadata
Client can verify read-your-writes or causal guarantees

Practice

(1/5)
1. In the request flow of a global load balancing system, at Step 3 the GSLB receives a DNS query for a user in Europe. What does the GSLB do at this step?
easy
A. Returns the IP of the closest data center without checking health status
B. Checks data center health and latency metrics before returning the IP address
C. Forwards the DNS query to the local data center's DNS server for resolution
D. Caches the previous DNS response indefinitely to reduce load

Solution

  1. Step 1: Understand GSLB behavior at DNS query

    GSLB does not blindly return IPs (Returns the IP of the closest data center without checking health status) or forward queries (Forwards the DNS query to the local data center's DNS server for resolution). It actively checks health and latency to pick the best endpoint.

  2. Step 2: Evaluate caching behavior

    Indefinite caching (Caches the previous DNS response indefinitely to reduce load) would prevent dynamic failover and load balancing.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    GSLB checks health and latency before DNS response [check]
Hint: GSLB checks health before DNS response [check]
Common Mistakes:
2. For a mobile app with limited control over HTTP headers and caching proxies, which API versioning strategy is best to ensure backward compatibility and cache friendliness?
medium
A. Embed version in the URL path (e.g., /v1/resource) to leverage caching and simple routing
B. Use query parameters to specify version, allowing flexible client control
C. Use a custom HTTP header like X-API-Version to specify version
D. Use content negotiation with Accept header media types for versioning

Solution

  1. Step 1: Consider mobile client and caching constraints

    Mobile clients and caching proxies often ignore or strip custom headers, making header-based versioning unreliable.

  2. Step 2: Evaluate URL versioning benefits

    Embedding version in URL ensures cache keys differ per version, enabling backward compatibility and cache friendliness.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    URL versioning works best with caching proxies and limited header control [check]
Hint: URL versioning beats headers for caching and mobile clients [check]
Common Mistakes:
3. For a high-throughput TCP service requiring minimal latency and no application-level inspection, which load balancing approach is best?
medium
A. Use a Layer 7 load balancer to route based on HTTP headers for flexibility
B. Use a Layer 4 load balancer to forward packets based on IP and port
C. Use DNS round-robin to distribute client requests evenly
D. Use an application proxy that terminates connections and inspects payload

Solution

  1. Step 1: Identify latency and throughput constraints

    Layer 4 load balancers operate at transport layer with minimal overhead, suitable for high throughput and low latency.

  2. Step 2: Eliminate options adding unnecessary overhead

    Layer 7 load balancers and application proxies inspect payload, increasing latency. DNS round-robin lacks health checks and session affinity.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Layer 4 LB minimizes latency by forwarding packets without payload inspection [check]
Hint: Minimal latency? Choose Layer 4 load balancing [check]
Common Mistakes:
4. At 10x scale, which component in the real-time leaderboard system is most likely to fail first, and why?
hard
A. The Redis cluster because sorted sets require in-memory storage and cannot shard scores efficiently
B. The API Gateway because it cannot handle the surge in client connections
C. The Ranking Worker because it cannot process the increased event stream fast enough
D. The database because it cannot handle the increased write throughput

Solution

  1. Step 1: Analyze Redis Sorted Sets scaling limits

    Sorted Sets require in-memory storage and have limited sharding options, making Redis cluster memory the bottleneck.

  2. Step 2: Compare other components

    Databases can be scaled horizontally; Ranking Workers can be parallelized; API Gateways scale with load balancers.

  3. Step 3: Identify Redis cluster as first failure point

    Memory pressure and inability to shard sorted sets efficiently cause Redis cluster to fail first under 10x load.

  4. Final Answer:

    Option A -> Option A

  5. Quick Check:

    Redis Sorted Sets memory and sharding limits cause first failure at scale [check]
Hint: Redis Sorted Sets memory limits break first at scale [check]
Common Mistakes:
5. At 10x scale, which component in WhatsApp's architecture is most likely to fail first and why?
hard
A. Chat Servers because WebSocket connections are stateful and cannot be resharded easily
B. Kafka brokers because partition count cannot be increased dynamically
C. The database because it cannot handle 10x more writes without sharding
D. Presence service because TTL cache invalidations become too frequent

Solution

  1. Step 1: Analyze stateful WebSocket connections on Chat Servers

    Chat Servers maintain in-memory state per WebSocket; scaling requires connection migration which is complex.

  2. Step 2: Compare other components

    Databases can shard; Kafka partitions can be increased with rebalancing; presence TTL cache scales horizontally.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Stateful WebSocket connections -> Chat Servers bottleneck at scale [check]
Hint: Stateful WebSocket -> Chat Servers bottleneck at 10x scale [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard