Data replication strategies in HLD - System Design Exercise

Design: Data Replication System

Design focuses on replication strategies and architecture for database systems. Does not cover detailed database schema design or application-level logic.

Functional Requirements

FR1: Replicate data across multiple database nodes to improve availability and fault tolerance

FR2: Support both synchronous and asynchronous replication modes

FR3: Ensure data consistency according to the chosen replication strategy

FR4: Allow read scaling by directing read requests to replicas

FR5: Handle failover automatically in case of primary node failure

FR6: Support recovery and catch-up of lagging replicas

Non-Functional Requirements

NFR1: System must support up to 1000 write transactions per second

NFR2: Replication latency should be under 100ms for synchronous mode

NFR3: Availability target of 99.9% uptime

NFR4: System should tolerate network partitions and node failures gracefully

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

Primary (master) database node

Replica (slave) database nodes

Replication log or write-ahead log (WAL)

Replication coordinator or manager

Failover detection and leader election mechanism

Monitoring and alerting system

Design Patterns

Master-slave replication

Multi-master replication

Synchronous vs asynchronous replication

Quorum-based replication

Log shipping and streaming replication

Conflict resolution strategies

Reference Architecture

          +---------------------+
          |     Application     |
          +----------+----------+
                     |
                     v
          +---------------------+          +---------------------+
          |   Primary Database   |<-------->| Replication Manager  |
          +----------+----------+          +----------+----------+
                     |                               |
        (Write-Ahead Log/WAL)                        |
                     |                               |
          +----------v----------+          +---------v----------+
          |   Replica Database   |          |   Replica Database  |
          +---------------------+          +---------------------+

Components

Primary Database

Relational or NoSQL DB with WAL support

Handles all write operations and generates replication logs

Replica Database

Same as primary

Receives and applies replication logs to stay in sync for reads and failover

Replication Manager

Custom or built-in DB component

Coordinates replication, manages log shipping, monitors lag and health

Failover Mechanism

Leader election tools like ZooKeeper or Raft

Detects primary failure and promotes a replica to primary

Request Flow

1. 1. Application sends write request to Primary Database.

2. 2. Primary writes data and records changes in Write-Ahead Log (WAL).

3. 3. Replication Manager streams WAL entries to Replica Databases.

4. 4. Replica Databases apply changes from WAL to update their data.

5. 5. Application read requests can be served from Replica Databases to reduce load on Primary.

6. 6. Failover Mechanism monitors Primary health; if failure detected, promotes a Replica to Primary.

7. 7. Lagging replicas catch up by replaying missing WAL entries.

Database Schema

Entities: None specific to replication; replication uses database transaction logs (WAL). Relationships: Primary node streams WAL to multiple Replica nodes in 1:N fashion.

Scaling Discussion

Bottlenecks

Primary node write throughput limits overall system writes

Network bandwidth limits replication log shipping speed

Replication lag increases with distance and load

Failover detection delay can increase downtime

Conflict resolution complexity in multi-master setups

Solutions

Scale primary vertically or shard data to distribute writes

Use compression and efficient protocols for log shipping

Deploy replicas closer to clients for read scaling and reduce lag

Implement fast leader election algorithms and health checks

Use conflict-free data types or application-level conflict resolution in multi-master

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain trade-offs between synchronous and asynchronous replication

Discuss consistency vs availability considerations

Describe how replication logs (WAL) enable data synchronization

Highlight failover and recovery mechanisms

Mention scaling challenges and solutions

Use simple diagrams to illustrate data flow