Replication lag monitoring in Redis - Step-by-Step Execution

Concept Flow - Replication lag monitoring

Master Redis Server

↓

Send replication stream

↓

Replica Redis Server

↓

Measure replication offset

↓

Calculate lag = Master offset - Replica offset

↓

Alert if lag > threshold

↓

Take action or monitor

This flow shows how Redis master sends data to replica, replica tracks offsets, and lag is calculated by comparing offsets to monitor replication delay.

Execution Sample

Redis

redis-cli info replication
redis-cli role
redis-cli --raw info replication | grep master_repl_offset
redis-cli --raw info replication | grep slave_repl_offset

These commands check replication info and offsets on master and replica to monitor replication lag.

Execution Table

Step	Command	Master Offset	Replica Offset	Lag Calculation	Action
1	redis-cli info replication on master	1200000	-	-	Get master offset
2	redis-cli info replication on replica	-	1199950	-	Get replica offset
3	Calculate lag	1200000	1199950	1200000 - 1199950 = 50	Lag = 50 bytes
4	Compare lag to threshold (e.g. 100)	1200000	1199950	50 < 100	No alert, replication healthy
5	Repeat monitoring periodically	Varies	Varies	Calculate lag each time	Continuous monitoring
6	If lag > threshold	Varies	Varies	Lag exceeds threshold	Trigger alert or action
7	Exit	-	-	-	Monitoring cycle ends or continues

💡 Monitoring stops or alerts when lag exceeds threshold or manual stop

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	Final
master_offset	0	1200000	1200000	1200000	1200000	Varies
replica_offset	0	0	1199950	1199950	1199950	Varies
lag	0	0	0	50	50	Varies

Key Moments - 3 Insights

Why do we subtract replica offset from master offset to get lag?

Because master offset shows total data sent, and replica offset shows data received. Their difference is the delay in bytes, as shown in execution_table step 3.

What does it mean if lag is zero or very small?

It means replica is almost caught up with master, replication is healthy. See execution_table step 4 where lag < threshold means no alert.

Why do we monitor lag repeatedly instead of once?

Replication lag can change over time due to network or load. Continuous monitoring (step 5) helps detect issues early.

Visual Quiz - 3 Questions

Look at the execution table, what is the lag value calculated at step 3?

A 1200000

B 1199950

C 50

D 100

💡 Hint

Check the 'Lag Calculation' column in row for step 3.

At which step does the system decide no alert is needed because lag is below threshold?

A Step 2

B Step 4

C Step 6

D Step 1

💡 Hint

Look at the 'Action' column where lag is compared to threshold.

If the replica offset was 1199900 instead of 1199950, how would lag change at step 3?

A Lag would be 100

B Lag would be 50

C Lag would be 0

D Lag would be negative

💡 Hint

Lag = master offset - replica offset; check variable_tracker values.

Concept Snapshot

Redis replication lag monitoring:
- Use 'info replication' to get master and replica offsets.
- Lag = master_repl_offset - replica_repl_offset.
- Monitor lag regularly.
- Alert if lag exceeds threshold.
- Helps ensure replica is up-to-date.

Full Transcript

Replication lag monitoring in Redis involves checking the data offsets on the master and replica servers. The master offset shows how much data has been sent, and the replica offset shows how much data has been received. By subtracting the replica offset from the master offset, we get the lag, which tells us how far behind the replica is. Commands like 'redis-cli info replication' help retrieve these offsets. Monitoring this lag regularly helps detect delays in replication. If the lag exceeds a set threshold, alerts can be triggered to take action. This process ensures the replica stays synchronized with the master.

Practice

1. What does the master_last_io_seconds_ago field in INFO replication indicate in Redis?

easy

A. The number of seconds since the master last communicated with the replica

B. The total number of connected replicas

C. The current memory usage of the master

D. The replication backlog size in bytes

Solution

1. Step 1: Understand the INFO replication output

   This command shows replication details including communication status between master and replicas.

2. Step 2: Interpret master_last_io_seconds_ago

   This field tells how many seconds ago the master last sent data to the replica, indicating replication lag.

3. Final Answer:

   The number of seconds since the master last communicated with the replica -> Option A

4. Quick Check:

   Replication lag = master_last_io_seconds_ago [OK]

Hint: Lag time is how long since last master-replica communication [OK]

Common Mistakes:

• Confusing lag with number of replicas
• Mixing memory usage with replication info
• Thinking backlog size shows lag

2. Which Redis command correctly shows replication lag details?

easy

A. SHOW replication

B. INFO replication

C. MONITOR replication

D. REPLICA STATUS

Solution

1. Step 1: Recall Redis commands for replication info

   The INFO command with the replication section shows replication status and lag.

2. Step 2: Verify command correctness

   MONITOR shows all commands, SHOW and REPLICA STATUS are invalid commands.

3. Final Answer:

   INFO replication -> Option B

4. Quick Check:

   Correct command = INFO replication [OK]

Hint: Use INFO replication to check lag, not monitor or show [OK]

Common Mistakes:

• Using MONITOR instead of INFO
• Trying non-existent commands
• Confusing SHOW with INFO

3. Given this INFO replication snippet:

master_link_status:up
master_last_io_seconds_ago:5

What does the master_last_io_seconds_ago value tell you?

medium

A. The replica is 5 seconds behind the master

B. The master has been down for 5 seconds

C. The master last sent data 5 seconds ago

D. There are 5 replicas connected

Solution

1. Step 1: Analyze master_link_status and master_last_io_seconds_ago

   The status 'up' means the master and replica are connected. The number 5 means 5 seconds since last data sent.

2. Step 2: Interpret the lag meaning

   The value shows how many seconds ago the master last communicated, not the exact lag or downtime.

3. Final Answer:

   The master last sent data 5 seconds ago -> Option C

4. Quick Check:

   master_last_io_seconds_ago = last data sent seconds [OK]

Hint: Lag means last data sent time, not downtime or replica count [OK]

Common Mistakes:

• Confusing lag with downtime
• Assuming number means replicas count
• Thinking it shows replica delay exactly

4. You run INFO replication and see master_link_status:down and master_last_io_seconds_ago:1000. What is the best first step to fix replication lag?

medium

A. Restart the Redis master server immediately

B. Flush all data on the replica

C. Increase the replication backlog size

D. Check network connectivity between master and replica

Solution

1. Step 1: Understand the meaning of master_link_status:down

   This means the replica lost connection to the master, causing lag.

2. Step 2: Identify the cause and fix

   Network issues are the common cause; checking connectivity is the first step before restarting or changing configs.

3. Final Answer:

   Check network connectivity between master and replica -> Option D

4. Quick Check:

   Link down means network check first [OK]

Hint: Link down? Check network before restarting or flushing [OK]

Common Mistakes:

• Restarting master without checking network
• Changing backlog size without cause
• Flushing replica data unnecessarily

5. You want to alert if replication lag exceeds 10 seconds. Which approach using INFO replication is best?

hard

A. Parse master_last_io_seconds_ago and trigger alert if > 10

B. Check connected_slaves count and alert if less than 1

C. Monitor master_link_status only for 'down' status

D. Use INFO memory to check memory usage spikes

Solution

1. Step 1: Identify the lag metric

   master_last_io_seconds_ago directly shows how many seconds since last master-replica communication, ideal for lag monitoring.

2. Step 2: Set alert condition

   Trigger alert when this value exceeds 10 seconds to catch lag early.

3. Final Answer:

   Parse master_last_io_seconds_ago and trigger alert if > 10 -> Option A

4. Quick Check:

   Lag alert = master_last_io_seconds_ago > threshold [OK]

Hint: Alert on master_last_io_seconds_ago > threshold seconds [OK]

Common Mistakes:

• Using connected_slaves count for lag
• Only watching link status without lag value
• Checking memory instead of replication lag