Data replication strategies in HLD - Scalability & System Analysis

Scalability Analysis - Data replication strategies

Growth Table: Data Replication Strategies

Users / Data Size	100 Users	10K Users	1M Users	100M Users
Data Volume	Small (MBs to GBs)	Medium (GBs to TBs)	Large (TBs to PBs)	Very Large (PBs+)
Replication Type	Simple master-slave replication	Asynchronous replication with read replicas	Multi-region replication with conflict resolution	Geo-distributed multi-master replication with sharding
Latency Tolerance	Low latency acceptable	Some replication lag tolerated	Eventual consistency common	Strong consistency challenging, eventual consistency preferred
Replication Lag	Minimal, near real-time	Seconds to minutes	Minutes to hours	Hours, depending on region and network
Failure Handling	Manual failover possible	Automated failover with monitoring	Conflict detection and resolution needed	Complex conflict resolution and partition tolerance

First Bottleneck

At small scale, the primary database server becomes the bottleneck due to write load and replication overhead.

As users grow, replication lag increases causing stale reads and inconsistent data views.

Network bandwidth limits cross-region replication speed at large scale.

Conflict resolution complexity grows with multi-master setups, impacting performance and consistency.

Scaling Solutions

• Horizontal scaling: Add read replicas to distribute read traffic and reduce load on master.
• Asynchronous replication: Use async replication to reduce write latency but accept eventual consistency.
• Multi-region replication: Deploy replicas in multiple regions to reduce latency for global users.
• Sharding: Partition data by key ranges or user segments to distribute load and storage.
• Conflict resolution: Implement application-level or database-level conflict handling for multi-master replication.
• Compression and batching: Compress data and batch replication updates to optimize network usage.
• Monitoring and automated failover: Use monitoring tools to detect failures and switch replicas automatically.

Back-of-Envelope Cost Analysis

• Requests per second (RPS):
  • 100 users: ~10-100 RPS
  • 10K users: ~1K-10K RPS
  • 1M users: ~100K-1M RPS
  • 100M users: ~10M+ RPS
• Storage needed:
  • 100 users: GBs
  • 10K users: TBs
  • 1M users: PBs
  • 100M users: Multiple PBs to Exabytes
• Network bandwidth:
  • Replication traffic grows with data volume and update frequency.
  • Cross-region replication requires high bandwidth and low latency links.

Interview Tip

Start by clarifying the scale and consistency requirements.

Discuss trade-offs between synchronous and asynchronous replication.

Explain how replication affects latency, availability, and consistency.

Describe how you would detect and handle replication lag and conflicts.

Outline scaling steps as user base grows, focusing on bottlenecks and solutions.

Self Check

Your database handles 1000 QPS. Traffic grows 10x to 10,000 QPS. What do you do first?

Answer: Add read replicas and implement connection pooling to distribute read load and reduce master bottleneck. Consider asynchronous replication to reduce write latency and monitor replication lag closely.

Key Result

Data replication strategies must evolve from simple master-slave setups at small scale to multi-region, multi-master, and sharded architectures at large scale to handle increased load, latency, and consistency challenges.

Practice

1. Which data replication strategy ensures that all copies of data are updated at the same time before confirming a write operation?

easy

A. Synchronous replication

B. Asynchronous replication

C. Master-slave replication

D. Multi-master replication

Solution

1. Step 1: Understand synchronous replication

   Synchronous replication waits for all copies to confirm the write before completing the operation.

2. Step 2: Compare with other strategies

   Asynchronous replication does not wait, master-slave and multi-master describe topology, not timing.

3. Final Answer:

   Synchronous replication -> Option A

4. Quick Check:

   Strong consistency = Synchronous replication [OK]

Hint: Strong consistency means waiting for all copies to update [OK]

Common Mistakes:

• Confusing asynchronous with synchronous replication
• Mixing replication timing with replication topology
• Assuming master-slave means synchronous

2. Which of the following is the correct description of asynchronous replication?

easy

A. Replicas confirm the data before the primary confirms the write

B. Data is copied to replicas without waiting for confirmation from them

C. Data is copied simultaneously to all replicas with confirmation

D. Data is copied to replicas before the primary confirms the write

Solution

1. Step 1: Define asynchronous replication

   It copies data to replicas but does not wait for their confirmation before confirming the write.

2. Step 2: Eliminate other options

   Replicas confirm the data before the primary confirms the write requires replica confirmation before primary ack (synchronous); B copies before primary ack (incorrect); C describes synchronous behavior.

3. Final Answer:

   Data is copied to replicas without waiting for confirmation from them -> Option B

4. Quick Check:

   Asynchronous = no wait for replica confirmation [OK]

Hint: Asynchronous means no waiting for replicas to confirm [OK]

Common Mistakes:

• Thinking asynchronous waits for replicas
• Confusing timing of data copy
• Mixing synchronous and asynchronous definitions

3. Consider a system using master-slave replication. If the master receives a write request, what happens next?

medium

A. The write is applied simultaneously to master and slaves synchronously

B. The write is applied to all slaves first, then to the master

C. The write is only applied to slaves, not the master

D. The write is applied to the master and then asynchronously copied to slaves

Solution

1. Step 1: Understand master-slave replication flow

   Writes go to the master first, then are copied to slaves, usually asynchronously.

2. Step 2: Check other options

   Slaves do not receive writes first or alone; synchronous writes to all are not typical in master-slave.

3. Final Answer:

   The write is applied to the master and then asynchronously copied to slaves -> Option D

4. Quick Check:

   Master handles writes first in master-slave [OK]

Hint: Master handles writes first, slaves get updates later [OK]

Common Mistakes:

• Assuming slaves can accept writes first
• Confusing synchronous with asynchronous replication
• Thinking slaves are independent masters

4. A system using multi-master replication faces conflicts when two masters update the same data simultaneously. What is a common way to fix this issue?

medium

A. Switch to asynchronous replication only

B. Disable writes on all but one master

C. Use conflict resolution strategies like timestamps or version vectors

D. Remove all replicas and use a single database

Solution

1. Step 1: Identify conflict in multi-master replication

   Simultaneous writes cause conflicts that need resolution strategies.

2. Step 2: Evaluate conflict resolution methods

   Timestamps or version vectors help decide which write wins or how to merge.

3. Final Answer:

   Use conflict resolution strategies like timestamps or version vectors -> Option C

4. Quick Check:

   Conflict resolution = timestamps/version vectors [OK]

Hint: Resolve conflicts with timestamps or version vectors [OK]

Common Mistakes:

• Disabling multi-master writes defeats purpose
• Thinking asynchronous replication solves conflicts
• Removing replicas is impractical

5. You are designing a global application with users worldwide needing fast reads and writes. Which replication strategy best fits this use case?

hard

A. Multi-master replication with conflict resolution

B. Master-slave replication with synchronous writes

C. Single master with asynchronous replication

D. No replication, single database instance

Solution

1. Step 1: Analyze global user needs

   Users worldwide require fast reads and writes from multiple locations.

2. Step 2: Match replication strategy

   Multi-master allows writes at multiple locations with conflict resolution, improving speed and availability.

3. Step 3: Eliminate other options

   Master-slave with synchronous writes is slow globally; single master limits write locations; no replication is slow and risky.

4. Final Answer:

   Multi-master replication with conflict resolution -> Option A

5. Quick Check:

   Global fast writes = multi-master + conflict resolution [OK]

Hint: Global writes need multi-master with conflict handling [OK]

Common Mistakes:

• Choosing synchronous master-slave for global writes
• Ignoring conflict resolution in multi-master
• Using single master limits write speed globally