How to Reassign Partitions in Kafka: Step-by-Step Guide

To reassign partitions in Kafka, use the kafka-reassign-partitions.sh tool with a JSON file that specifies the new partition assignments. Run the tool with --execute to apply changes and --verify to check progress.

Syntax

The kafka-reassign-partitions.sh script is used to move partitions between brokers. It requires a JSON file describing the new assignments.

Basic command structure:

• --zookeeper <zookeeper_host:port>: Connects to Zookeeper managing Kafka metadata.
• --reassignment-json-file <file_path>: Path to JSON file with new partition assignments.
• --execute: Applies the reassignment.
• --verify: Checks the status of the reassignment.

bash

kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file reassignment.json --execute

Example

This example shows how to move partitions of a topic named my-topic from broker 1 to brokers 2 and 3.

First, create a JSON file reassignment.json with the new assignments.

json/bash

{
  "partitions": [
    {
      "topic": "my-topic",
      "partition": 0,
      "replicas": [2, 3]
    },
    {
      "topic": "my-topic",
      "partition": 1,
      "replicas": [3, 2]
    }
  ],
  "version": 1
}

# Command to execute reassignment
kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file reassignment.json --execute

# Command to verify reassignment
kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file reassignment.json --verify

Output

Starting partition reassignment for topic 'my-topic'...
Reassignment started successfully.

Verification output:
Status: In Progress
Partition 0: Moving from broker 1 to brokers 2,3
Partition 1: Moving from broker 1 to brokers 3,2

Once complete, status will show 'Completed'.

Common Pitfalls

• Incorrect JSON format: The JSON must be valid and include partitions and version keys.
• Not verifying reassignment: Always run with --verify to ensure reassignment completes.
• Using wrong Zookeeper address: Use the correct Zookeeper host and port managing your Kafka cluster.
• Reassigning live partitions without caution: Reassignment can cause temporary unavailability; plan during low traffic.

json

Wrong JSON example:
{
  "topic": "my-topic",
  "partition": 0,
  "replicas": [2,3]
}

Correct JSON example:
{
  "partitions": [
    {
      "topic": "my-topic",
      "partition": 0,
      "replicas": [2,3]
    }
  ],
  "version": 1
}

Quick Reference

Command	Description
--execute	Apply the partition reassignment described in the JSON file
--verify	Check the status of the ongoing partition reassignment
--zookeeper	Specify Zookeeper connection for Kafka metadata
--reassignment-json-file	JSON file with new partition assignments

Key Takeaways

Use kafka-reassign-partitions.sh with a valid JSON file to reassign partitions.

Always verify reassignment progress with the --verify option.

Ensure the JSON file includes partitions array and version key.

Use the correct Zookeeper address managing your Kafka cluster.

Plan reassignment during low traffic to avoid service disruption.