In-sync replicas (ISR) in Kafka - Step-by-Step Execution

Process Flow - In-sync replicas (ISR)

Leader receives write request

↓

Leader writes to local log

↓

Leader sends data to followers

↓

Followers append data to their logs

↓

Followers send acknowledgment

↓

Leader updates ISR list

↓

Leader commits message if ISR quorum

↓

Next write request

This flow shows how the leader and followers keep data in sync by exchanging writes and acknowledgments to maintain the ISR list.

Execution Sample

Kafka

Leader receives write
Leader writes locally
Leader sends to followers
Followers append and ack
Leader updates ISR
Leader commits message

Shows the step-by-step process of how replicas stay in sync and how the ISR list is updated.

Process Table

Step	Action	Leader Log	Follower Logs	ISR List	Commit Status
1	Leader receives write request	[]	[]	[]	No
2	Leader writes data locally	[msg1]	[]	[]	No
3	Leader sends data to followers	[msg1]	[]	[]	No
4	Followers append data and ack	[msg1]	[msg1]	[]	No
5	Leader updates ISR list	[msg1]	[msg1]	[Leader, Follower1]	No
6	Leader commits message (quorum ack)	[msg1]	[msg1]	[Leader, Follower1]	Yes
7	Next write request arrives	[msg1]	[msg1]	[Leader, Follower1]	Yes

💡 Process stops here waiting for next write or follower lag

Status Tracker

Variable	Start	After Step 2	After Step 4	After Step 5	After Step 6
Leader Log	[]	[msg1]	[msg1]	[msg1]	[msg1]
Follower Logs	[]	[]	[msg1]	[msg1]	[msg1]
ISR List	[]	[]	[]	[Leader, Follower1]	[Leader, Follower1]
Commit Status	No	No	No	No	Yes

Key Moments - 2 Insights

Why does the ISR list update only after followers acknowledge?

Because the ISR list includes only replicas that have confirmed they have the latest data, as shown in step 5 where ISR updates after follower acks in step 4.

What happens if a follower does not acknowledge in time?

The follower is removed from the ISR list, so the leader will not wait for its ack to commit messages, ensuring availability (not shown in this trace but implied after step 5).

Visual Quiz - 3 Questions

Look at the execution table, at which step does the leader commit the message?

A Step 4

B Step 6

C Step 5

D Step 7

💡 Hint

Check the 'Commit Status' column in the execution table where it changes to 'Yes'.

At which step do followers append the message to their logs?

A Step 4

B Step 2

C Step 3

D Step 5

💡 Hint

Look at the 'Follower Logs' column where it changes from empty to '[msg1]'.

If a follower fails to acknowledge, how would the ISR list change after step 5?

A ISR list would be empty

B ISR list would include all replicas

C ISR list would exclude the non-acknowledging follower

D ISR list would not change

💡 Hint

Refer to the key moment about follower acknowledgments affecting ISR list.

Concept Snapshot

In-sync replicas (ISR) track replicas that have fully caught up with the leader.
Leader writes data locally, sends to followers.
Followers append data and send acknowledgments.
Leader updates ISR list with replicas that acked.
Leader commits messages once ISR quorum is reached.
ISR ensures data durability and availability.

Full Transcript

This visual trace shows how Kafka's In-sync replicas (ISR) mechanism works. The leader receives a write request and writes the data to its local log. Then it sends the data to follower replicas. Followers append the data to their logs and send acknowledgments back to the leader. Once the leader receives acknowledgments from followers, it updates the ISR list to include those replicas. When a quorum of ISR members have acknowledged, the leader commits the message, making it durable. If a follower does not acknowledge in time, it is removed from the ISR list to maintain availability. This process repeats for each new write request, ensuring data consistency and fault tolerance.

Practice

1. What does In-sync replicas (ISR) mean in Kafka?

easy

A. Replicas that are offline

B. Replicas that never receive data

C. Replicas that have the latest data from the leader

D. Replicas that only store old data

Solution

1. Step 1: Understand the role of ISR

   ISR are replicas that keep up-to-date with the leader's data.

2. Step 2: Compare options

   Only Replicas that have the latest data from the leader correctly describes ISR as replicas with the latest data.

3. Final Answer:

   Replicas that have the latest data from the leader -> Option C

4. Quick Check:

   ISR = Latest data replicas [OK]

Hint: ISR means replicas fully caught up with leader data [OK]

Common Mistakes:

• Thinking ISR includes offline replicas
• Confusing ISR with replicas that lag behind
• Assuming ISR means replicas with old data

2. Which Kafka configuration controls the minimum number of ISR replicas required to acknowledge a write?

easy

A. replica.fetch.max.bytes

B. min.insync.replicas

C. max.insync.replicas

D. offsets.retention.minutes

Solution

1. Step 1: Identify configuration related to ISR acknowledgments

   The setting min.insync.replicas defines the minimum ISR count for write success.

2. Step 2: Eliminate unrelated options

   Other options control fetch size or retention, not ISR acknowledgments.

3. Final Answer:

   min.insync.replicas -> Option B

4. Quick Check:

   Minimum ISR replicas = min.insync.replicas [OK]

Hint: min.insync.replicas sets minimum ISR for writes [OK]

Common Mistakes:

• Confusing min.insync.replicas with max.insync.replicas
• Choosing unrelated configs like offsets.retention.minutes
• Assuming replica.fetch.max.bytes controls ISR count

3. Given a Kafka topic with 3 replicas and ISR = [replica1, replica2], what happens if replica2 falls behind and leaves ISR?

medium

A. Only replica1 remains in ISR; leader continues with one ISR

B. Leader stops producing until replica2 catches up

C. All replicas are removed from ISR

D. Replica2 becomes the new leader immediately

Solution

1. Step 1: Understand ISR behavior when a replica falls behind

   If a replica lags, it is removed from ISR, leaving others in ISR.

2. Step 2: Analyze leader and ISR state

   The leader continues with remaining ISR replicas; it does not stop or change leader immediately.

3. Final Answer:

   Only replica1 remains in ISR; leader continues with one ISR -> Option A

4. Quick Check:

   Lagging replica leaves ISR, leader continues [OK]

Hint: Lagging replica leaves ISR; leader stays with remaining ISR [OK]

Common Mistakes:

• Thinking leader stops producing when ISR shrinks
• Assuming all replicas leave ISR together
• Believing lagging replica becomes leader immediately

4. You see this error in Kafka logs: "Not enough replicas are in ISR to satisfy min.insync.replicas". What is the likely cause?

medium

A. The consumer group is not committed

B. The leader replica is offline

C. The topic has zero partitions

D. Too few replicas are currently in ISR compared to min.insync.replicas

Solution

1. Step 1: Interpret the error message

   The error means the number of replicas in ISR is less than the configured minimum.

2. Step 2: Match cause with options

   Too few replicas are currently in ISR compared to min.insync.replicas directly matches the error cause; other options are unrelated to ISR count.

3. Final Answer:

   Too few replicas are currently in ISR compared to min.insync.replicas -> Option D

4. Quick Check:

   ISR count below min.insync.replicas causes error [OK]

Hint: Error means ISR count < min.insync.replicas [OK]

Common Mistakes:

• Blaming leader offline instead of ISR count
• Confusing consumer group commit with ISR errors
• Thinking zero partitions cause this error

5. You want to ensure high availability in Kafka by allowing leader failover only if at least 2 replicas are in ISR. Which configuration and setup achieve this?

hard

A. Set min.insync.replicas=2 and have at least 3 replicas per partition

B. Set min.insync.replicas=1 and have 2 replicas per partition

C. Set min.insync.replicas=3 with 2 replicas per partition

D. Set min.insync.replicas=0 and 1 replica per partition

Solution

1. Step 1: Understand min.insync.replicas meaning

   This setting defines the minimum ISR count required for leader failover and write success.

2. Step 2: Match configuration to requirement

   To require at least 2 ISR replicas, set min.insync.replicas=2 and have at least 3 replicas to allow failover.

3. Final Answer:

   Set min.insync.replicas=2 and have at least 3 replicas per partition -> Option A

4. Quick Check:

   High availability needs min.insync.replicas=2 with 3 replicas [OK]

Hint: min.insync.replicas ≤ replica count for failover [OK]

Common Mistakes:

• Setting min.insync.replicas higher than replica count
• Using too few replicas for desired availability
• Setting min.insync.replicas to zero disables safety