Database replication (master-slave) in HLD - System Design Guide

Problem Statement

When a single database server handles all reads and writes, it becomes a bottleneck causing slow response times under heavy load. If this server fails, the entire system loses data availability, leading to downtime and potential data loss.

Solution

Database replication creates copies of the primary database (master) on one or more secondary servers (slaves). The master handles all writes and propagates changes to slaves, which serve read requests. This spreads read load and provides failover options if the master fails.

Architecture

Client

→Master

↓

→Slave2

This diagram shows clients sending writes to the master database, which replicates data asynchronously or synchronously to multiple slave databases that handle read requests.

Trade-offs

✓ Pros

→

Improves read scalability by distributing read traffic across multiple slaves.

→

Provides high availability through failover to slave if master fails.

→

Reduces load on master, improving write performance stability.

→

Enables backup and analytics on slaves without impacting master.

✗ Cons

→

Replication lag can cause slaves to serve stale data temporarily.

→

Write operations still bottleneck on the single master node.

→

Complexity in handling failover and ensuring data consistency.

→

Increased infrastructure and maintenance overhead.

Use when read traffic significantly exceeds write traffic (e.g., read-to-write ratio > 5:1) and high availability is required for read operations.

Avoid when write traffic is very high and requires horizontal scaling, or when strict real-time consistency between reads and writes is mandatory.

Real World Examples

Twitter

Uses master-slave replication to scale read-heavy timelines while writes go to the master, balancing load and improving availability.

GitHub

Employs master-slave replication to separate read queries from write operations, ensuring fast repository browsing without impacting commit performance.

Shopify

Uses replication to offload reporting and analytics queries to slaves, preventing impact on transactional master database.

Alternatives

Multi-master replication

Allows multiple nodes to accept writes and replicate changes to each other, unlike master-slave where only one node accepts writes.

Use when: Choose when write availability and horizontal write scaling are critical, and conflict resolution mechanisms are in place.

Sharding

Splits data horizontally across multiple databases, each responsible for a subset of data, rather than replicating the same data across nodes.

Use when: Choose when dataset size or write throughput exceeds single database capacity.

Summary

Master-slave replication prevents a single database server from becoming a read bottleneck by distributing reads to replicas.

It improves availability by allowing failover to slaves if the master fails, but writes remain centralized on the master.

Replication lag and write bottlenecks are key trade-offs to consider when using this pattern.

Practice

1. What is the primary role of the master in a master-slave database replication setup?

easy

A. Handle all write operations and propagate changes to slaves

B. Serve read requests only

C. Backup data from slaves

D. Monitor network traffic between nodes

Solution

1. Step 1: Understand master responsibilities

   The master database is responsible for handling all write operations to ensure data consistency.

2. Step 2: Understand slave responsibilities

   Slaves replicate data from the master and typically handle read requests to reduce load on the master.

3. Final Answer:

   Handle all write operations and propagate changes to slaves -> Option A

4. Quick Check:

   Master = writes + replication [OK]

Hint: Master handles writes; slaves handle reads [OK]

Common Mistakes:

• Thinking slaves handle writes
• Confusing backup role with master
• Assuming master only monitors

2. Which of the following is the correct sequence for setting up master-slave replication?

easy

A. Configure slaves only, master is optional

B. Configure slaves, then configure master, then start replication

C. Start replication, then configure master and slaves

D. Configure master, then configure slaves, then start replication

Solution

1. Step 1: Configure master first

   The master must be configured first to accept writes and log changes for replication.

2. Step 2: Configure slaves after master

   Slaves are then configured to connect to the master and replicate its data.

3. Final Answer:

   Configure master, then configure slaves, then start replication -> Option D

4. Quick Check:

   Master setup before slaves [OK]

Hint: Master setup precedes slaves setup [OK]

Common Mistakes:

• Configuring slaves before master
• Starting replication before configuration
• Ignoring master configuration

3. Consider a master-slave setup where the master receives 100 writes per second and each slave replicates with a 2-second lag. If a read request is sent to a slave immediately after a write, what is the likely result?

medium

A. The slave returns the latest data including the write

B. The slave crashes due to replication lag

C. The slave returns stale data missing the latest write

D. The slave rejects the read request

Solution

1. Step 1: Understand replication lag effect

   Replication lag means slaves are behind the master by 2 seconds, so recent writes are not yet applied.

2. Step 2: Analyze read request timing

   A read immediately after a write will hit the slave before replication applies the change, so stale data is returned.

3. Final Answer:

   The slave returns stale data missing the latest write -> Option C

4. Quick Check:

   Replication lag causes stale reads [OK]

Hint: Replication lag means slaves have delayed data [OK]

Common Mistakes:

• Assuming slaves always have latest data
• Thinking slaves reject reads during lag
• Believing slaves crash due to lag

4. A system uses master-slave replication but experiences inconsistent reads from slaves. Which change can fix this issue?

medium

A. Allow writes on slaves to sync data faster

B. Increase monitoring and reduce replication lag

C. Disable replication and use only master for reads

D. Add more slaves without changing replication

Solution

1. Step 1: Identify cause of inconsistent reads

   Inconsistent reads usually happen due to replication lag causing slaves to have outdated data.

2. Step 2: Fix by reducing replication lag

   Improving monitoring and optimizing replication reduces lag, making slave reads more consistent.

3. Final Answer:

   Increase monitoring and reduce replication lag -> Option B

4. Quick Check:

   Reduce lag to fix inconsistent reads [OK]

Hint: Reduce lag to fix stale slave reads [OK]

Common Mistakes:

• Allowing writes on slaves breaks consistency
• Disabling replication loses read scaling
• Adding slaves without lag fix doesn't help

5. You are designing a high-availability system using master-slave replication. Which approach best ensures minimal downtime and data loss during master failure?

hard

A. Automatically promote a slave to master and redirect writes

B. Stop all writes until the master is manually restored

C. Allow writes on slaves during master downtime

D. Use only one master without slaves to avoid complexity

Solution

1. Step 1: Understand master failure impact

   If the master fails, writes cannot proceed unless a new master is available.

2. Step 2: Use automatic failover

   Promoting a slave to master automatically and redirecting writes ensures availability and minimal data loss.

3. Step 3: Evaluate other options

   Stopping writes causes downtime; writing on slaves breaks consistency; single master lacks redundancy.

4. Final Answer:

   Automatically promote a slave to master and redirect writes -> Option A

5. Quick Check:

   Automatic failover ensures availability [OK]

Hint: Auto-promote slave for failover [OK]

Common Mistakes:

• Stopping writes causes downtime
• Writing on slaves breaks data integrity
• Ignoring failover leads to outages