Leader election in Kafka - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is leader election in Kafka?

Leader election is the process Kafka uses to choose a broker as the leader for a partition. This leader handles all reads and writes for that partition to keep data consistent.

beginner

Why is leader election important in Kafka?

Leader election ensures high availability and fault tolerance. If a leader broker fails, Kafka quickly elects a new leader so clients can continue reading and writing without interruption.

intermediate

Which Kafka component manages leader election?

Kafka uses Apache ZooKeeper (or its own KRaft mode in newer versions) to manage leader election. ZooKeeper keeps track of brokers and helps select leaders for partitions.

intermediate

What happens if a Kafka leader broker goes down?

When a leader broker goes down, Kafka detects the failure and triggers a new leader election for the affected partitions. A new broker from the in-sync replicas becomes the leader.

advanced

How does Kafka ensure data consistency during leader election?

Kafka only elects a leader from brokers that have fully caught up with the leader's data (in-sync replicas). This prevents data loss and keeps consistency.

What role does the Kafka leader broker play?

A Handles all reads and writes for its partition

B Only stores backup data

C Manages ZooKeeper nodes

D Distributes messages to clients

The leader broker manages all reads and writes for its partition to ensure consistency.

Which system does Kafka use for leader election in older versions?

A Kafka Streams

B ZooKeeper

C KRaft

D Kafka Connect

Kafka uses ZooKeeper to manage leader election in older versions.

What triggers a new leader election in Kafka?

A Message consumption

B Client connection

C New topic creation

D Leader broker failure

A new leader election happens when the current leader broker fails.

From which brokers does Kafka elect a new leader?

A Only the oldest broker

B Any broker randomly

C In-sync replicas

D Only brokers with no data

Kafka elects a new leader only from brokers that are in-sync replicas to maintain data consistency.

What is the main goal of leader election in Kafka?

A Maintain availability and consistency

B Increase message size

C Reduce network traffic

D Encrypt messages

Leader election helps keep Kafka available and consistent even if brokers fail.

Explain how leader election works in Kafka and why it is important.

Think about what happens when a broker stops working.

Describe the role of in-sync replicas in Kafka's leader election process.

Focus on data synchronization and safety.

Practice

1. What is the main purpose of leader election in Kafka?

easy

A. To backup data to external storage

B. To balance the load evenly among all brokers

C. To choose one broker to lead and coordinate tasks

D. To monitor client connections

Solution

1. Step 1: Understand leader election role

   Leader election picks one broker as the leader to manage partition tasks.

2. Step 2: Compare options

   Only To choose one broker to lead and coordinate tasks describes choosing a broker to lead and coordinate tasks, which matches Kafka's leader election purpose.

3. Final Answer:

   To choose one broker to lead and coordinate tasks -> Option C

4. Quick Check:

   Leader election = choose leader broker [OK]

Hint: Leader election picks one leader broker, not all brokers [OK]

Common Mistakes:

• Confusing leader election with load balancing
• Thinking leader election backs up data
• Assuming leader election monitors clients

2. Which of the following is the correct way to configure Kafka broker IDs for leader election?

easy

A. broker.id=unique_integer_per_broker

B. broker.id=localhost

C. broker.id=auto_assign

D. broker.id=0.0.0.0

Solution

1. Step 1: Identify broker ID type

   Kafka requires each broker to have a unique integer ID for leader election.

2. Step 2: Check options

   Only broker.id=unique_integer_per_broker uses a unique integer, while others use invalid or non-integer values.

3. Final Answer:

   broker.id=unique_integer_per_broker -> Option A

4. Quick Check:

   Broker ID must be unique integer [OK]

Hint: Broker ID must be a unique integer, not a string or IP [OK]

Common Mistakes:

• Using hostnames or IPs as broker IDs
• Setting broker.id to zero or non-unique values
• Assuming auto assignment works without config

3. Given the following Kafka cluster state, which broker will be the leader for partition 0?

Broker 1: ISR = [1, 2, 3]
Broker 2: ISR = [1, 2, 3]
Broker 3: ISR = [1, 2, 3]
Current leader: Broker 2
Broker 2 fails

medium

A. Broker 2

B. Broker 1

C. Broker 3

D. No leader assigned

Solution

1. Step 1: Understand ISR and leader failover

   ISR (In-Sync Replicas) are eligible leaders. When current leader fails, next ISR broker becomes leader.

2. Step 2: Identify next leader after Broker 2 fails

   Broker 1 and 3 are in ISR. Kafka picks the lowest broker ID in ISR as new leader, so Broker 1.

3. Final Answer:

   Broker 1 -> Option B

4. Quick Check:

   Next leader = lowest ISR broker after failure [OK]

Hint: Leader election picks lowest ISR broker if current leader fails [OK]

Common Mistakes:

• Choosing failed broker as leader
• Picking broker outside ISR
• Assuming no leader assigned after failure

4. You have a Kafka cluster where leader election is not working properly. Which of the following is the most likely cause?

medium

A. Each broker has a unique broker.id

B. Zookeeper is running and connected

C. Partitions have multiple replicas

D. All brokers have the same broker.id

Solution

1. Step 1: Identify leader election requirements

   Leader election requires each broker to have a unique broker.id to distinguish them.

2. Step 2: Analyze causes

   If all brokers share the same broker.id, leader election cannot assign a unique leader, causing failure.

3. Final Answer:

   All brokers have the same broker.id -> Option D

4. Quick Check:

   Unique broker IDs needed for leader election [OK]

Hint: Check broker IDs; duplicates break leader election [OK]

Common Mistakes:

• Assuming Zookeeper connection causes failure
• Thinking multiple replicas cause election failure
• Ignoring broker.id uniqueness

5. In a Kafka cluster using KRaft mode (without Zookeeper), how does leader election differ from the traditional Zookeeper-based approach?

hard

A. KRaft uses an internal quorum of controllers to elect leaders without external Zookeeper

B. KRaft requires manual leader assignment by the admin

C. KRaft disables leader election and uses round-robin partition assignment

D. KRaft uses Zookeeper but with a different election algorithm

Solution

1. Step 1: Understand KRaft mode leader election

   KRaft mode replaces Zookeeper with an internal controller quorum that manages leader election.

2. Step 2: Compare options

   Only KRaft uses an internal quorum of controllers to elect leaders without external Zookeeper correctly states KRaft uses internal quorum without external Zookeeper for leader election.

3. Final Answer:

   KRaft uses an internal quorum of controllers to elect leaders without external Zookeeper -> Option A

4. Quick Check:

   KRaft = internal quorum leader election [OK]

Hint: KRaft replaces Zookeeper with internal controller quorum [OK]

Common Mistakes:

• Thinking KRaft disables leader election
• Assuming manual leader assignment in KRaft
• Believing KRaft still uses external Zookeeper