Introduction

Kafka topics are where messages are stored and organized. Configuring topics lets you control how data is kept, how long it stays, and how it is shared among users.

When you want to set how many copies of your data Kafka keeps for safety.

When you need to decide how long messages stay before they are deleted.

When you want to control how many parts a topic has to allow multiple users to read and write at the same time.

When you want to change the way Kafka cleans up old messages to save space.

When you want to adjust the speed and size of data Kafka handles for better performance.

Commands

This command creates a new Kafka topic named 'example-topic' with 3 partitions and 2 copies of data for safety. It also sets messages to be kept for 7 days (604800000 milliseconds).

Terminal

kafka-topics.sh --create --topic example-topic --bootstrap-server localhost:9092 --partitions 3 --replication-factor 2 --config retention.ms=604800000

Expected OutputExpected

Created topic example-topic.

→

--partitions - Sets how many parts the topic has for parallel processing

→

--replication-factor - Sets how many copies of data Kafka keeps

→

--config retention.ms - Sets how long messages stay before deletion

This command shows the details of the 'example-topic' to check its configuration and status.

Terminal

kafka-topics.sh --describe --topic example-topic --bootstrap-server localhost:9092

Expected OutputExpected

Topic: example-topic PartitionCount: 3 ReplicationFactor: 2 Configs: retention.ms=604800000 Topic: example-topic Partition: 0 Leader: 1 Replicas: 1,2 Isr: 1,2 Topic: example-topic Partition: 1 Leader: 2 Replicas: 2,1 Isr: 2,1 Topic: example-topic Partition: 2 Leader: 1 Replicas: 1,2 Isr: 1,2

This command changes the retention time of 'example-topic' to 3 days (259200000 milliseconds) after creation.

Terminal

kafka-configs.sh --alter --entity-type topics --entity-name example-topic --add-config retention.ms=259200000 --bootstrap-server localhost:9092

Expected OutputExpected

No output (command runs silently)

→

--add-config - Adds or changes a configuration for the topic

This command checks the updated configuration of 'example-topic' to confirm the retention time change.

Terminal

kafka-topics.sh --describe --topic example-topic --bootstrap-server localhost:9092

Expected OutputExpected

Topic: example-topic PartitionCount: 3 ReplicationFactor: 2 Configs: retention.ms=259200000 Topic: example-topic Partition: 0 Leader: 1 Replicas: 1,2 Isr: 1,2 Topic: example-topic Partition: 1 Leader: 2 Replicas: 2,1 Isr: 2,1 Topic: example-topic Partition: 2 Leader: 1 Replicas: 1,2 Isr: 1,2

Key Concept

If you remember nothing else from this pattern, remember: Kafka topic configurations control how data is stored, copied, and cleaned up to fit your needs.

Common Mistakes

Creating a topic without specifying replication-factor or partitions.

Kafka uses defaults that may not fit your needs, risking data loss or poor performance.

Always specify replication-factor and partitions explicitly when creating topics.

Changing topic configurations without using the correct command or flags.

Changes won't apply and the topic will keep old settings.

Use kafka-configs.sh with --alter and --add-config flags to update topic settings.

Setting retention.ms to a very low value without understanding impact.

Messages may be deleted too soon, causing data loss.

Set retention.ms carefully based on how long you need to keep messages.

Summary

Use kafka-topics.sh with --create to make a topic with partitions, replication, and configs.

Use kafka-topics.sh with --describe to check topic details and confirm settings.

Use kafka-configs.sh with --alter and --add-config to change topic configurations after creation.