Controller broker in Kafka - Commands & Configuration

Introduction

In Kafka, the controller broker is a special broker that manages cluster-wide actions like topic partition leadership and broker membership. It helps keep the cluster organized and running smoothly by coordinating these tasks.

When a Kafka broker starts and needs to know if it should become the controller.

When a broker fails and leadership of partitions must be reassigned.

When new brokers join the cluster and the cluster metadata needs updating.

When partitions are created or deleted and leadership must be assigned.

When you want to monitor which broker is currently the controller for troubleshooting.

Commands

Check the API versions supported by the Kafka broker to ensure it is running and reachable.

Terminal

kafka-broker-api-versions.sh --bootstrap-server localhost:9092

Expected Output

1.0.0

→

--bootstrap-server - Specifies the Kafka broker address to connect to.

List all topics with their partition leaders and replicas to see which broker is leading partitions.

Terminal

kafka-topics.sh --bootstrap-server localhost:9092 --describe

Expected Output

Topic: my-topic	PartitionCount: 3	ReplicationFactor: 2	Configs: 

	Topic: my-topic	Partition: 0	Leader: 1	Replicas: 1,2	Isr: 1,2

	Topic: my-topic	Partition: 1	Leader: 2	Replicas: 2,3	Isr: 2,3

	Topic: my-topic	Partition: 2	Leader: 1	Replicas: 1,3	Isr: 1,3

→

--bootstrap-server - Connects to the Kafka broker.

→

--describe - Shows detailed information about topics.

Display the current controller broker ID to know which broker is managing the cluster.

Terminal

kafka-run-class.sh kafka.admin.ControllerState --bootstrap-server localhost:9092

Expected Output

Controller ID: 1

→

--bootstrap-server - Connects to the Kafka broker.

Key Concept

If you remember nothing else, remember: the controller broker is the leader broker that manages cluster-wide decisions like partition leadership and broker membership.

Common Mistakes

Trying to identify the controller by checking any broker's logs without using the controller state command.

Broker logs may not clearly indicate controller status, causing confusion.

Use the kafka.admin.ControllerState command to directly query the current controller broker.

Assuming the controller broker never changes after cluster startup.

The controller can change if the current controller fails or is shut down.

Regularly check the controller broker especially after broker restarts or failures.

Summary

Use kafka.admin.ControllerState to find the current controller broker ID.

Describe topics to see which brokers lead partitions managed by the controller.

The controller broker manages cluster-wide tasks like leadership and membership.