What is ISR in Kafka: Explained Simply

In Kafka, ISR stands for In-Sync Replicas, which are the set of replicas that are fully caught up with the leader's data. ISR ensures data durability and consistency by tracking replicas that have the latest messages and can safely serve reads or take over if the leader fails.

How It Works

Imagine a classroom where the teacher (leader) writes notes on the board, and several students (replicas) copy them down. The In-Sync Replicas (ISR) are the students who have copied all the notes up to the current point without falling behind. If a student misses some notes or is too slow, they are temporarily removed from this group.

In Kafka, the leader broker writes data to its log and followers replicate this data. The ISR is the group of follower replicas that have fully caught up with the leader's log. Kafka uses ISR to decide which replicas are healthy and can safely serve data or become the new leader if the current leader fails. This mechanism helps keep data consistent and prevents data loss.

Example

This example shows how to check the ISR for a Kafka topic partition using the Kafka command-line tool.

bash

kafka-topics.sh --describe --topic my-topic --bootstrap-server localhost:9092

Output

Topic: my-topic	Partition: 0	Leader: 1	Replicas: 1,2,3	Isr: 1,2

When to Use

Use ISR to monitor the health and synchronization status of Kafka replicas. It is crucial when you want to ensure high availability and data durability in your Kafka cluster. For example, if a replica falls out of the ISR, it means it is lagging and might not have the latest data, so it should not be used as a leader until it catches up.

In real-world scenarios, ISR helps in failover situations where the leader broker crashes. Kafka promotes one of the in-sync replicas to leader, ensuring no data loss. It also helps in tuning replication and acknowledgment settings to balance performance and reliability.

Key Points

• ISR means replicas fully caught up with the leader.
• Only replicas in ISR can become leaders during failover.
• ISR helps prevent data loss by tracking replica synchronization.
• Monitoring ISR helps detect slow or failing replicas.

Key Takeaways

ISR stands for In-Sync Replicas, the replicas fully caught up with the leader.

Kafka uses ISR to ensure data consistency and safe leader election.

Replicas outside ISR are considered lagging and not eligible for leadership.

Monitoring ISR helps maintain cluster health and data durability.

ISR is essential for fault tolerance and high availability in Kafka.