In-sync replicas (ISR) in Kafka - Commands & Configuration

Introduction

In Kafka, In-sync replicas (ISR) are the set of replicas that are fully caught up with the leader. They help keep data safe by ensuring copies of messages are stored on multiple brokers before confirming writes.

When you want to ensure no data loss if a broker fails by having multiple copies of data.

When you want to monitor which replicas are healthy and up-to-date with the leader.

When you want to configure how many replicas must confirm a write before it is considered successful.

When troubleshooting replication lag or broker failures in a Kafka cluster.

When tuning Kafka for high availability and durability of messages.

Commands

This command shows detailed information about the topic 'example-topic', including the leader, replicas, and the in-sync replicas (ISR). It helps verify which replicas are currently in sync.

Terminal

kafka-topics.sh --describe --topic example-topic --bootstrap-server localhost:9092

Expected Output

Topic: example-topic	PartitionCount: 1	ReplicationFactor: 3	Configs: 

	Topic: example-topic	Partition: 0	Leader: 1	Replicas: 1,2,3	Isr: 1,2,3

→

--describe - Shows detailed information about the topic.

→

--topic - Specifies the topic to describe.

→

--bootstrap-server - Specifies the Kafka server to connect to.

This command displays the configuration of the topic 'example-topic', including settings related to replication and ISR behavior.

Terminal

kafka-configs.sh --bootstrap-server localhost:9092 --entity-type topics --entity-name example-topic --describe

Expected Output

Configs for topic 'example-topic' are: 

min.insync.replicas=2

retention.ms=604800000

→

--entity-type topics - Specifies that the entity is a topic.

→

--entity-name example-topic - Specifies the topic name.

→

--describe - Shows the current configuration.

This command updates the topic 'example-topic' to require at least 2 replicas to be in sync before acknowledging writes, improving data durability.

Terminal

kafka-topics.sh --alter --topic example-topic --config min.insync.replicas=2 --bootstrap-server localhost:9092

Expected Output

No output (command runs silently)

→

--alter - Modifies the topic configuration.

→

--config min.insync.replicas=2 - Sets the minimum number of in-sync replicas required.

Verify the updated configuration and check the current ISR after the change.

Terminal

kafka-topics.sh --describe --topic example-topic --bootstrap-server localhost:9092

Expected Output

Topic: example-topic	PartitionCount: 1	ReplicationFactor: 3	Configs: min.insync.replicas=2

	Topic: example-topic	Partition: 0	Leader: 1	Replicas: 1,2,3	Isr: 1,2,3

→

--describe - Shows detailed topic information.

Key Concept

If you remember nothing else, remember: In-sync replicas are the copies of data that are fully caught up with the leader and ensure safe, durable writes.

Common Mistakes

Setting min.insync.replicas higher than the replication factor.

This causes Kafka to reject all writes because it can never meet the required number of in-sync replicas.

Set min.insync.replicas to a value less than or equal to the replication factor.

Ignoring ISR status and assuming all replicas are always in sync.

Leads to data loss risk if some replicas lag or fail without detection.

Regularly check ISR status with kafka-topics.sh --describe to monitor replica health.

Summary

Use kafka-topics.sh --describe to see the leader, replicas, and in-sync replicas of a topic.

Configure min.insync.replicas to control how many replicas must confirm writes for durability.

Monitor ISR to ensure replicas are healthy and data is safely replicated.

Practice

1. What does In-sync replicas (ISR) mean in Kafka?

easy

A. Replicas that are offline

B. Replicas that never receive data

C. Replicas that have the latest data from the leader

D. Replicas that only store old data

Solution

1. Step 1: Understand the role of ISR

   ISR are replicas that keep up-to-date with the leader's data.

2. Step 2: Compare options

   Only Replicas that have the latest data from the leader correctly describes ISR as replicas with the latest data.

3. Final Answer:

   Replicas that have the latest data from the leader -> Option C

4. Quick Check:

   ISR = Latest data replicas [OK]

Hint: ISR means replicas fully caught up with leader data [OK]

Common Mistakes:

• Thinking ISR includes offline replicas
• Confusing ISR with replicas that lag behind
• Assuming ISR means replicas with old data

2. Which Kafka configuration controls the minimum number of ISR replicas required to acknowledge a write?

easy

A. replica.fetch.max.bytes

B. min.insync.replicas

C. max.insync.replicas

D. offsets.retention.minutes

Solution

1. Step 1: Identify configuration related to ISR acknowledgments

   The setting min.insync.replicas defines the minimum ISR count for write success.

2. Step 2: Eliminate unrelated options

   Other options control fetch size or retention, not ISR acknowledgments.

3. Final Answer:

   min.insync.replicas -> Option B

4. Quick Check:

   Minimum ISR replicas = min.insync.replicas [OK]

Hint: min.insync.replicas sets minimum ISR for writes [OK]

Common Mistakes:

• Confusing min.insync.replicas with max.insync.replicas
• Choosing unrelated configs like offsets.retention.minutes
• Assuming replica.fetch.max.bytes controls ISR count

3. Given a Kafka topic with 3 replicas and ISR = [replica1, replica2], what happens if replica2 falls behind and leaves ISR?

medium

A. Only replica1 remains in ISR; leader continues with one ISR

B. Leader stops producing until replica2 catches up

C. All replicas are removed from ISR

D. Replica2 becomes the new leader immediately

Solution

1. Step 1: Understand ISR behavior when a replica falls behind

   If a replica lags, it is removed from ISR, leaving others in ISR.

2. Step 2: Analyze leader and ISR state

   The leader continues with remaining ISR replicas; it does not stop or change leader immediately.

3. Final Answer:

   Only replica1 remains in ISR; leader continues with one ISR -> Option A

4. Quick Check:

   Lagging replica leaves ISR, leader continues [OK]

Hint: Lagging replica leaves ISR; leader stays with remaining ISR [OK]

Common Mistakes:

• Thinking leader stops producing when ISR shrinks
• Assuming all replicas leave ISR together
• Believing lagging replica becomes leader immediately

4. You see this error in Kafka logs: "Not enough replicas are in ISR to satisfy min.insync.replicas". What is the likely cause?

medium

A. The consumer group is not committed

B. The leader replica is offline

C. The topic has zero partitions

D. Too few replicas are currently in ISR compared to min.insync.replicas

Solution

1. Step 1: Interpret the error message

   The error means the number of replicas in ISR is less than the configured minimum.

2. Step 2: Match cause with options

   Too few replicas are currently in ISR compared to min.insync.replicas directly matches the error cause; other options are unrelated to ISR count.

3. Final Answer:

   Too few replicas are currently in ISR compared to min.insync.replicas -> Option D

4. Quick Check:

   ISR count below min.insync.replicas causes error [OK]

Hint: Error means ISR count < min.insync.replicas [OK]

Common Mistakes:

• Blaming leader offline instead of ISR count
• Confusing consumer group commit with ISR errors
• Thinking zero partitions cause this error

5. You want to ensure high availability in Kafka by allowing leader failover only if at least 2 replicas are in ISR. Which configuration and setup achieve this?

hard

A. Set min.insync.replicas=2 and have at least 3 replicas per partition

B. Set min.insync.replicas=1 and have 2 replicas per partition

C. Set min.insync.replicas=3 with 2 replicas per partition

D. Set min.insync.replicas=0 and 1 replica per partition

Solution

1. Step 1: Understand min.insync.replicas meaning

   This setting defines the minimum ISR count required for leader failover and write success.

2. Step 2: Match configuration to requirement

   To require at least 2 ISR replicas, set min.insync.replicas=2 and have at least 3 replicas to allow failover.

3. Final Answer:

   Set min.insync.replicas=2 and have at least 3 replicas per partition -> Option A

4. Quick Check:

   High availability needs min.insync.replicas=2 with 3 replicas [OK]

Hint: min.insync.replicas ≤ replica count for failover [OK]

Common Mistakes:

• Setting min.insync.replicas higher than replica count
• Using too few replicas for desired availability
• Setting min.insync.replicas to zero disables safety