ZooKeeper role (and KRaft replacement) in Kafka - Mini Project: Build & Apply

ZooKeeper Role and KRaft Replacement in Kafka

📖 Scenario: You are managing a Kafka cluster that currently uses ZooKeeper for metadata management. Kafka is moving towards using KRaft mode, which replaces ZooKeeper. You want to understand how to configure and verify the Kafka cluster roles in both ZooKeeper and KRaft modes.

🎯 Goal: Learn how to set up Kafka metadata management with ZooKeeper and then switch to KRaft mode, understanding the role of each component and verifying the cluster state.

📋 What You'll Learn

Create a Kafka configuration file with ZooKeeper connection

Add a configuration variable to enable KRaft mode

Write a command to check the Kafka cluster metadata in ZooKeeper mode

Write a command to check the Kafka cluster metadata in KRaft mode

💡 Why This Matters

🌍 Real World

Kafka clusters currently use ZooKeeper for metadata, but newer versions use KRaft mode to simplify architecture and improve scalability.

💼 Career

Understanding Kafka metadata management is essential for Kafka administrators and DevOps engineers managing streaming data infrastructure.

1

Create Kafka configuration with ZooKeeper connection

Create a file named server.properties with the following exact content:
zookeeper.connect=localhost:2181
This sets up Kafka to use ZooKeeper for metadata management.

Kafka

# Create a file named server.properties with ZooKeeper connection
# Your code here

Need a hint?

Use the exact line zookeeper.connect=localhost:2181 in the configuration file.

2

Add KRaft mode configuration

Add these exact lines to server.properties to enable KRaft mode:
process.roles=controller,broker
node.id=1
controller.quorum.voters=1@localhost:9093
This configures Kafka to run in KRaft mode without ZooKeeper.

Kafka

zookeeper.connect=localhost:2181
# Add KRaft mode configuration lines below
# Your code here

Need a hint?

Include all three lines exactly as shown to enable KRaft mode.

3

Check Kafka cluster metadata in ZooKeeper mode

Write the exact command to describe Kafka topics using ZooKeeper mode:
kafka-topics.sh --describe --zookeeper localhost:2181
This command shows the metadata managed by ZooKeeper.

Kafka

# Write the command to describe topics in ZooKeeper mode
# Your code here

Need a hint?

Use the --zookeeper localhost:2181 option to query ZooKeeper metadata.

4

Check Kafka cluster metadata in KRaft mode

Write the exact command to describe Kafka topics using KRaft mode:
kafka-topics.sh --describe --bootstrap-server localhost:9092
This command shows the metadata managed by KRaft controllers.

Kafka

# Write the command to describe topics in KRaft mode
# Your code here

Need a hint?

Use the --bootstrap-server localhost:9092 option to query KRaft metadata.