In-sync replicas (ISR) in Kafka - Mini Project: Build & Apply

In-sync Replicas (ISR) in Kafka

📖 Scenario: You are managing a Kafka cluster that replicates messages across multiple brokers to keep data safe. You want to understand which replicas are currently in-sync with the leader to ensure data reliability.

🎯 Goal: Build a simple Kafka script that lists the in-sync replicas (ISR) for a given topic partition.

📋 What You'll Learn

Create a dictionary called topic_partitions with one entry: key 'my_topic-0' and value {'replicas': [1, 2, 3], 'leader': 1, 'isr': [1, 2]}

Create a variable called partition_key and set it to 'my_topic-0'

Use a for loop with variable replica to iterate over the replicas and check if each is in the ISR list

Create a list called replica_status that stores strings like 'Replica 1 is in-sync' or 'Replica 3 is out-of-sync'

Print each string in replica_status on a new line

💡 Why This Matters

🌍 Real World

Kafka uses in-sync replicas to ensure data is safely replicated across brokers. Monitoring ISR helps maintain data durability and availability.

💼 Career

Understanding ISR is important for roles like Kafka administrators, DevOps engineers, and backend developers working with distributed systems.

1

Create the topic partition data

Create a dictionary called topic_partitions with one entry: key 'my_topic-0' and value {'replicas': [1, 2, 3], 'leader': 1, 'isr': [1, 2]}

Kafka

# Create the topic_partitions dictionary with the given key and value
# Your code here

Hint

Think of topic_partitions as a map where the key is the topic and partition, and the value is another map with replica info.

2

Set the partition key

Create a variable called partition_key and set it to 'my_topic-0'

Kafka

topic_partitions = {'my_topic-0': {'replicas': [1, 2, 3], 'leader': 1, 'isr': [1, 2]}}
# Create the partition_key variable and assign 'my_topic-0'
# Your code here

Hint

This variable will help us access the data for the specific partition.

3

Check which replicas are in-sync

Use a for loop with variable replica to iterate over the replicas in topic_partitions[partition_key]['replicas'] and check if each replica is in topic_partitions[partition_key]['isr']. Create a list called replica_status that stores strings like 'Replica 1 is in-sync' or 'Replica 3 is out-of-sync'

Kafka

topic_partitions = {'my_topic-0': {'replicas': [1, 2, 3], 'leader': 1, 'isr': [1, 2]}}
partition_key = 'my_topic-0'
# Create replica_status list and fill it by checking each replica's sync status
# Your code here

Hint

Loop over each replica and check if it is in the ISR list. Use an f-string to create the status message.

4

Print the replica sync status

Print each string in the replica_status list on a new line using a for loop

Kafka

topic_partitions = {'my_topic-0': {'replicas': [1, 2, 3], 'leader': 1, 'isr': [1, 2]}}
partition_key = 'my_topic-0'
replica_status = []
for replica in topic_partitions[partition_key]['replicas']:
    in_sync = replica in topic_partitions[partition_key]['isr']
    status = f"Replica {replica} is {'in-sync' if in_sync else 'out-of-sync'}"
    replica_status.append(status)
# Print each replica status on a new line
# Your code here

Hint

Use a simple for loop to print each message in the list.

Practice

1. What does In-sync replicas (ISR) mean in Kafka?

easy

A. Replicas that are offline

B. Replicas that never receive data

C. Replicas that have the latest data from the leader

D. Replicas that only store old data

Solution

1. Step 1: Understand the role of ISR

   ISR are replicas that keep up-to-date with the leader's data.

2. Step 2: Compare options

   Only Replicas that have the latest data from the leader correctly describes ISR as replicas with the latest data.

3. Final Answer:

   Replicas that have the latest data from the leader -> Option C

4. Quick Check:

   ISR = Latest data replicas [OK]

Hint: ISR means replicas fully caught up with leader data [OK]

Common Mistakes:

• Thinking ISR includes offline replicas
• Confusing ISR with replicas that lag behind
• Assuming ISR means replicas with old data

2. Which Kafka configuration controls the minimum number of ISR replicas required to acknowledge a write?

easy

A. replica.fetch.max.bytes

B. min.insync.replicas

C. max.insync.replicas

D. offsets.retention.minutes

Solution

1. Step 1: Identify configuration related to ISR acknowledgments

   The setting min.insync.replicas defines the minimum ISR count for write success.

2. Step 2: Eliminate unrelated options

   Other options control fetch size or retention, not ISR acknowledgments.

3. Final Answer:

   min.insync.replicas -> Option B

4. Quick Check:

   Minimum ISR replicas = min.insync.replicas [OK]

Hint: min.insync.replicas sets minimum ISR for writes [OK]

Common Mistakes:

• Confusing min.insync.replicas with max.insync.replicas
• Choosing unrelated configs like offsets.retention.minutes
• Assuming replica.fetch.max.bytes controls ISR count

3. Given a Kafka topic with 3 replicas and ISR = [replica1, replica2], what happens if replica2 falls behind and leaves ISR?

medium

A. Only replica1 remains in ISR; leader continues with one ISR

B. Leader stops producing until replica2 catches up

C. All replicas are removed from ISR

D. Replica2 becomes the new leader immediately

Solution

1. Step 1: Understand ISR behavior when a replica falls behind

   If a replica lags, it is removed from ISR, leaving others in ISR.

2. Step 2: Analyze leader and ISR state

   The leader continues with remaining ISR replicas; it does not stop or change leader immediately.

3. Final Answer:

   Only replica1 remains in ISR; leader continues with one ISR -> Option A

4. Quick Check:

   Lagging replica leaves ISR, leader continues [OK]

Hint: Lagging replica leaves ISR; leader stays with remaining ISR [OK]

Common Mistakes:

• Thinking leader stops producing when ISR shrinks
• Assuming all replicas leave ISR together
• Believing lagging replica becomes leader immediately

4. You see this error in Kafka logs: "Not enough replicas are in ISR to satisfy min.insync.replicas". What is the likely cause?

medium

A. The consumer group is not committed

B. The leader replica is offline

C. The topic has zero partitions

D. Too few replicas are currently in ISR compared to min.insync.replicas

Solution

1. Step 1: Interpret the error message

   The error means the number of replicas in ISR is less than the configured minimum.

2. Step 2: Match cause with options

   Too few replicas are currently in ISR compared to min.insync.replicas directly matches the error cause; other options are unrelated to ISR count.

3. Final Answer:

   Too few replicas are currently in ISR compared to min.insync.replicas -> Option D

4. Quick Check:

   ISR count below min.insync.replicas causes error [OK]

Hint: Error means ISR count < min.insync.replicas [OK]

Common Mistakes:

• Blaming leader offline instead of ISR count
• Confusing consumer group commit with ISR errors
• Thinking zero partitions cause this error

5. You want to ensure high availability in Kafka by allowing leader failover only if at least 2 replicas are in ISR. Which configuration and setup achieve this?

hard

A. Set min.insync.replicas=2 and have at least 3 replicas per partition

B. Set min.insync.replicas=1 and have 2 replicas per partition

C. Set min.insync.replicas=3 with 2 replicas per partition

D. Set min.insync.replicas=0 and 1 replica per partition

Solution

1. Step 1: Understand min.insync.replicas meaning

   This setting defines the minimum ISR count required for leader failover and write success.

2. Step 2: Match configuration to requirement

   To require at least 2 ISR replicas, set min.insync.replicas=2 and have at least 3 replicas to allow failover.

3. Final Answer:

   Set min.insync.replicas=2 and have at least 3 replicas per partition -> Option A

4. Quick Check:

   High availability needs min.insync.replicas=2 with 3 replicas [OK]

Hint: min.insync.replicas ≤ replica count for failover [OK]

Common Mistakes:

• Setting min.insync.replicas higher than replica count
• Using too few replicas for desired availability
• Setting min.insync.replicas to zero disables safety