ZooKeeper role (and KRaft replacement) in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this Kafka cluster state query?

Consider a Kafka cluster running with ZooKeeper. You run the command to describe the cluster state and get the following output snippet:

Controller ID: 1
Broker IDs: 1, 2, 3
ZooKeeper connected: true

What does this output tell you about the cluster?

A The cluster has three brokers, broker 3 is the controller, and ZooKeeper is connected.

B The cluster has one broker, ZooKeeper is disconnected, and broker 1 is the controller.

C The cluster has three brokers, broker 1 is the controller, and ZooKeeper is connected.

D The cluster has three brokers, no controller is elected, and ZooKeeper is disconnected.

💡 Hint

Look for the Controller ID and ZooKeeper connection status in the output.

The output shows Controller ID as 1, meaning broker 1 is the controller. Broker IDs list three brokers. ZooKeeper connected is true, so ZooKeeper is active.

🧠 Conceptual

intermediate

What is the main role of ZooKeeper in a Kafka cluster?

In Kafka clusters that use ZooKeeper, what is ZooKeeper primarily responsible for?

A Storing all Kafka messages permanently.

B Managing cluster metadata, leader election, and configuration storage.

C Handling client requests and producing messages.

D Running Kafka Connect and Kafka Streams applications.

💡 Hint

Think about what coordination tasks are needed in a distributed system.

ZooKeeper manages metadata like broker info, topic configurations, and elects leaders for partitions. It does not store messages or handle client requests directly.

❓ Predict Output

advanced

What error occurs when Kafka tries to start without ZooKeeper in legacy mode?

You start a Kafka broker configured to use ZooKeeper but the ZooKeeper service is down. The broker logs show:

ERROR Fatal error during KafkaServer startup. Exiting.
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss

What does this error mean?

A Kafka broker cannot connect to ZooKeeper, so it fails to start.

B Kafka broker started successfully without ZooKeeper.

C Kafka broker is running but cannot produce messages.

D Kafka broker lost connection to clients.

💡 Hint

Look at the exception type and message about connection loss.

The error shows Kafka cannot connect to ZooKeeper, which is required in legacy mode, so startup fails.

🧠 Conceptual

advanced

How does KRaft mode replace ZooKeeper in Kafka?

Kafka introduced KRaft mode to remove the dependency on ZooKeeper. What is the key difference in how KRaft manages metadata compared to ZooKeeper?

A KRaft uses an internal quorum of Kafka brokers to manage metadata without an external service.

B KRaft requires an external ZooKeeper cluster to run alongside Kafka brokers.

C KRaft stores metadata in a separate database outside Kafka and ZooKeeper.

D KRaft disables metadata management and relies on clients to handle it.

💡 Hint

Think about how Kafka can manage metadata internally without ZooKeeper.

KRaft mode replaces ZooKeeper by using a built-in consensus quorum among Kafka brokers to manage metadata internally.

🚀 Application

expert

What happens to Kafka cluster availability during KRaft controller failure?

In a Kafka cluster running in KRaft mode with 3 controller nodes, one controller node fails unexpectedly. What is the expected behavior regarding cluster availability and metadata management?

A Kafka automatically switches back to ZooKeeper mode to maintain availability.

B All brokers shut down because metadata is lost.

C The entire Kafka cluster becomes unavailable until the failed controller is restored.

D The remaining controllers continue managing metadata, and the cluster remains available without interruption.

💡 Hint

Consider how consensus and fault tolerance work in KRaft mode with multiple controllers.

KRaft uses a quorum of controllers. Losing one controller in a 3-node quorum still leaves a majority to manage metadata and keep the cluster running.

Practice

1. What is the main role of ZooKeeper in a Kafka cluster?

easy

A. To store Kafka messages permanently

B. To replace Kafka brokers in the cluster

C. To serve as a Kafka client for producers

D. To manage cluster coordination and metadata

Solution

1. Step 1: Understand ZooKeeper's purpose

   ZooKeeper is used to manage coordination tasks like leader election and metadata storage in Kafka clusters.

2. Step 2: Identify what ZooKeeper does not do

   ZooKeeper does not store Kafka messages or act as a client or broker replacement.

3. Final Answer:

   To manage cluster coordination and metadata -> Option D

4. Quick Check:

   ZooKeeper = cluster coordination [OK]

Hint: ZooKeeper manages coordination, not message storage [OK]

Common Mistakes:

• Thinking ZooKeeper stores Kafka messages
• Confusing ZooKeeper with Kafka brokers
• Assuming ZooKeeper acts as a client

2. Which of the following is the correct way to enable KRaft mode in Kafka's configuration?

easy

A. set kafka.broker.only=true

B. set kafka.zk.enabled=true

C. set process.roles=controller,broker

D. set zookeeper.connect=localhost:2181

Solution

1. Step 1: Identify KRaft configuration property

   KRaft mode requires setting the process.roles property to include controller and broker roles.

2. Step 2: Eliminate incorrect options

   ZooKeeper-related settings like kafka.zk.enabled=true and zookeeper.connect=localhost:2181 are not for KRaft. kafka.broker.only=true is not a valid Kafka config.

3. Final Answer:

   set process.roles=controller,broker -> Option C

4. Quick Check:

   process.roles = controller,broker [OK]

Hint: KRaft uses process.roles, not zookeeper.connect [OK]

Common Mistakes:

• Confusing ZooKeeper settings with KRaft
• Using deprecated or invalid config keys
• Assuming KRaft disables brokers

3. Given the following Kafka broker configuration snippet in KRaft mode:

process.roles=broker,controller
node.id=1
controller.quorum.voters=1@localhost:9093

What is the role of this broker?

medium

A. It acts as both broker and controller

B. It acts only as a controller, not a broker

C. It acts neither as broker nor controller

D. It acts only as a broker, not a controller

Solution

1. Step 1: Analyze process.roles value

   The value 'broker,controller' means this node acts as both broker and controller.

2. Step 2: Confirm node.id and quorum voters

   node.id=1 and controller.quorum.voters=1@localhost:9093 confirm this node is part of the controller quorum.

3. Final Answer:

   It acts as both broker and controller -> Option A

4. Quick Check:

   process.roles includes both roles [OK]

Hint: process.roles lists all roles this node performs [OK]

Common Mistakes:

• Ignoring multiple roles in process.roles
• Confusing node.id with role type
• Assuming controller.quorum.voters disables broker role

4. You try to start a Kafka cluster in KRaft mode but get an error about missing controller quorum voters. Which configuration is missing or incorrect?

medium

A. Incorrect 'process.roles' without 'broker'

B. Missing or incorrect 'controller.quorum.voters' setting

C. Missing 'zookeeper.connect' setting

D. Missing 'log.dirs' setting

Solution

1. Step 1: Understand error context

   KRaft mode requires 'controller.quorum.voters' to define controller nodes.

2. Step 2: Identify missing or wrong config

   If this setting is missing or wrong, Kafka cannot form the controller quorum, causing startup errors.

3. Final Answer:

   Missing or incorrect 'controller.quorum.voters' setting -> Option B

4. Quick Check:

   controller.quorum.voters required in KRaft [OK]

Hint: KRaft needs controller.quorum.voters set [OK]

Common Mistakes:

• Confusing ZooKeeper settings with KRaft
• Ignoring controller quorum config
• Assuming log.dirs causes quorum errors

5. You want to migrate a Kafka cluster from ZooKeeper mode to KRaft mode. Which of the following steps is essential for a successful migration?

hard

A. Configure 'process.roles' and 'controller.quorum.voters' and migrate metadata to KRaft

B. Remove all brokers and restart ZooKeeper only

C. Disable KRaft and keep using ZooKeeper indefinitely

D. Set 'zookeeper.connect' to empty string and restart brokers

Solution

1. Step 1: Understand migration requirements

   Moving to KRaft requires configuring roles and controller quorum voters and migrating metadata from ZooKeeper.

2. Step 2: Eliminate incorrect options

   The other options do not perform a proper migration to KRaft or incorrectly suggest keeping or modifying ZooKeeper.

3. Final Answer:

   Configure 'process.roles' and 'controller.quorum.voters' and migrate metadata to KRaft -> Option A

4. Quick Check:

   Migration needs roles, quorum, and metadata move [OK]

Hint: Set roles, quorum, migrate metadata for KRaft migration [OK]

Common Mistakes:

• Skipping metadata migration step
• Trying to run ZooKeeper and KRaft simultaneously
• Misconfiguring controller quorum voters