Bird
Raised Fist0

Data Replication Strategies - Synchronous vs Asynchronous, Quorum Writes

HLDmediumInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Data Replication Strategies - Synchronous vs Asynchronous, Quorum Writes
Designing data replication with synchronous, asynchronous, and quorum write strategies for consistency and availability at scale
mediumStorage SystemsFreq: high
⚙️
Functional Requirements
  • Support synchronous replication ensuring strong consistency across replicas
  • Support asynchronous replication for higher write throughput and availability
  • Implement quorum write strategy to balance consistency and availability
  • Allow configurable replication factor (e.g., 3 replicas per data shard)
  • Provide failover and recovery mechanisms for replica failures
  • Expose APIs to write and read data with consistency guarantees
📊
Non-Functional Requirements
  • Handle 2 billion daily active users generating 40 messages per user per day
  • Achieve 99.99% availability for writes and reads
  • Maintain write latency under 100ms P99 for synchronous writes
  • Support data retention of 30 days with replication
  • Ensure system scales to 925K write requests per second at peak
❓
Clarifying Questions to Ask
  • What is the replication factor and number of replicas per data shard?
  • What consistency levels are required for reads and writes?
  • Are there specific latency SLAs for synchronous vs asynchronous writes?
  • What failure scenarios must the system tolerate (node, network partitions)?
  • Is the system deployed in a single region or multi-region?
  • What is the expected data size per message and retention period?
Out of Scope
✗ Design of client-side caching or offline sync
✗ Detailed encryption or security mechanisms
✗ Cross-datacenter geo-replication beyond primary region
✗ Data schema design unrelated to replication
📌
Assumptions
  • 2 billion DAU
  • 40 messages per user per day
  • 100 bytes per message
  • Replication factor = 3
⚡
QPS Calculation
write qps2B × 40 / 86400 ≈ 925,926 writes/sec
peak write qpsAssuming 2× peak factor → 1,851,852 writes/sec
read qpsAssuming 3 reads per write → 2,777,778 reads/sec
💾
Storage Estimate
daily storage2B × 40 × 100 bytes = 8 TB/day
retention 30 days8 TB × 30 = 240 TB
with replication240 TB × 3 = 720 TB total storage
🖥️
Server Estimate
storage nodesAssuming 4 TB usable per node → 720 TB / 4 TB = 180 nodes
replication coordinatorAssuming 10K writes/sec per coordinator → 1,851,852 / 10K ≈ 186 coordinators
🏗️
Architecture Diagram
Architecture Diagram
          +----------------------+
          |      Client API      |
          +----------+-----------+
                     | Write/Read Requests
                     v
          +----------------------+
          | Replication Coordinator|
          +----------+-----------+
           |          |           |
           |          |           |
           v          v           v
   +---------+  +---------+  +---------+
   | Replica |  | Replica |  | Replica |
   |  Node1  |  |  Node2  |  |  Node3  |
   +---------+  +---------+  +---------+
           ^          ^           ^
           |          |           |
           +----------+-----------+
                     |
          +----------------------+
          |   Persistent Storage  |
          +----------------------+
🧩
Components
Replication Coordinator
Manages write requests, enforces replication strategy (sync/async/quorum), and tracks replica states
Why thisKafka Streams with custom logic for quorum writes enables high throughput coordination with fault tolerance
Why notZooKeeper is not designed for high QPS write coordination and adds latency overhead
Replica Nodes
Store replicated data shards and respond to read/write requests per replication protocol
Why thisApache Cassandra supports tunable consistency with quorum writes and scales horizontally to 180+ nodes
Why notPostgreSQL requires manual sharding and does not support quorum writes natively at 1.8M writes/sec
Persistent Storage
Durably stores data on SSD-backed volumes for fast writes and reads
Why thisNVMe SSDs provide low latency and high IOPS needed for 100ms P99 write latency
Why notHDDs have higher latency and cannot meet strict write latency SLAs
Client API Layer
Exposes write/read endpoints with consistency level parameters to clients
Why thisgRPC enables low-latency, strongly typed RPC calls with built-in retries and deadlines
Why notREST over HTTP adds higher latency and less control over consistency parameters
Monitoring & Alerting
Tracks replication lag, node health, and SLA compliance
Why thisPrometheus + Grafana provide real-time metrics and alerting for distributed systems
Why notCustom monitoring lacks ecosystem support and scalability for 180+ nodes
🔄
Data Flow
1
Client API → Replication Coordinator
Client sends write request with consistency level (sync/async/quorum)
Consistency level determines replication strategy
2
Replication Coordinator → Replica Nodes
Coordinator forwards write to all replicas
For synchronous, waits for all replicas; for quorum, waits for majority
3
Replica Nodes → Replication Coordinator
Replicas acknowledge write success or failure
Coordinator aggregates acknowledgments to decide write success
4
Replication Coordinator → Client API
Coordinator responds to client with success/failure
Write latency depends on replication strategy
5
Client API → Replica Nodes
Client issues read requests with consistency level
Reads served from replicas per requested consistency
6
Replica Nodes → Client API
Replicas return data or error if stale
Quorum reads ensure latest committed data
7
Monitoring & Alerting → Operations Team
Alerts on replication lag or node failures
Enables proactive failure handling

Practice

(1/5)
1. In the CI/CD pipeline request flow, at Step 5 the Deployment Worker receives a build artifact reference and triggers deployment. What happens if the artifact is missing from the artifact registry at this step?
easy
A. Deployment Worker proceeds with last known artifact version silently
B. Deployment Worker retries indefinitely until artifact appears
C. Deployment Worker fails deployment and reports an error to the orchestrator
D. Deployment Worker skips deployment and triggers rollback automatically

Solution

  1. Step 1: Trace Deployment Worker behavior on missing artifact

    The Deployment Worker validates artifact presence before deployment; missing artifact causes failure.

  2. Step 2: Check fallback behaviors

    Retrying indefinitely would block pipeline; proceeding silently risks inconsistency; automatic rollback is triggered only after failed deployment, not missing artifact.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Deployment Worker fails and reports error on missing artifact [check]
Hint: Missing artifact -> deployment fails, no silent fallback [check]
Common Mistakes:
2. In the URL shortener system, at step 5 the Redirect Service receives a short code and queries the Cache before the Database. What happens if the Cache lookup misses at this step?
easy
A. The Redirect Service returns a 404 error immediately without querying the Database
B. The Redirect Service queries the Database for the original URL, then updates the Cache with the result
C. The Redirect Service generates a new short code and stores it in the Cache
D. The Redirect Service queues the request for later processing by an asynchronous worker

Solution

  1. Step 1: Trace Redirect Service behavior at Cache miss

    Cache miss means the short code is not in memory; the service must fetch the original URL from persistent storage.

  2. Step 2: Confirm Cache update after DB fetch

    After retrieving from DB, the service updates the Cache to speed up future requests.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Cache miss triggers DB lookup and Cache update [check]
Hint: Cache miss -> DB lookup -> Cache update [check]
Common Mistakes:
3. In the request flow for a nearby search, at step 4 the Quadtree component receives a query bounding box. What does the Quadtree do at this step?
easy
A. It returns all businesses within the bounding box without further filtering
B. It converts the bounding box into Geohash prefixes and queries a key-value store
C. It recursively traverses nodes intersecting the bounding box to collect candidate locations
D. It sends the bounding box to a full-text search engine for keyword filtering

Solution

  1. Step 1: Understand Quadtree traversal

    At step 4, the Quadtree recursively visits nodes overlapping the bounding box to efficiently narrow down candidate locations.

  2. Step 2: Eliminate other options

    It returns all businesses within the bounding box without further filtering is incorrect because Quadtree does filtering, not a full scan. It converts the bounding box into Geohash prefixes and queries a key-value store describes Geohash usage, not Quadtree. It sends the bounding box to a full-text search engine for keyword filtering is unrelated to spatial indexing.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Quadtree recursively traverses nodes intersecting query box [check]
Hint: Quadtree recursively visits nodes overlapping query bounding box [OK]
Common Mistakes:
4. In a time-series ingestion pipeline, at step 4 the Aggregation Worker reads raw event batches from Kafka and writes summarized metrics to the time-series database. What happens if the Aggregation Worker crashes after reading but before writing?
easy
A. The raw events remain in Kafka and will be reprocessed after worker restart
B. The Aggregation Worker automatically retries writing without re-reading events
C. The summarized metrics are partially written causing data corruption in the database
D. The raw events are lost because Kafka deletes them immediately after consumption

Solution

  1. Step 1: Understand Kafka's offset commit behavior

    Kafka retains messages until consumers commit offsets. If the worker crashes before committing, messages remain available for reprocessing.

  2. Step 2: Analyze crash timing

    Since the worker crashed before writing summaries and committing offsets, raw events remain in Kafka and will be reprocessed after restart.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Kafka retains uncommitted messages for reprocessing [check]
Hint: Kafka commits offsets after processing to avoid data loss [check]
Common Mistakes:
5. For a multiplayer online game requiring real-time position updates with minimal delay, which transport protocol and design choice best balances speed and reliability?
medium
A. Use UDP without any reliability mechanism to maximize speed
B. Use TCP exclusively to ensure all packets arrive in order
C. Use UDP with application-level sequence numbers and selective retransmission
D. Use TCP with aggressive Nagle's algorithm tuning to reduce latency

Solution

  1. Step 1: Analyze game requirements

    Real-time position updates require low latency but some reliability to avoid inconsistent state.

  2. Step 2: Evaluate protocol trade-offs

    UDP provides low latency; adding application-level sequence numbers and selective retransmission balances speed and reliability.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    UDP + app-level reliability fits real-time game updates [check]
Hint: UDP + app reliability balances speed and correctness [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard