Topic deletion and cleanup in Kafka - Commands & Configuration

Introduction

Sometimes you need to remove old or unused Kafka topics to keep your system clean and avoid wasting storage. Deleting topics helps manage resources and keeps your Kafka environment organized.

When a topic is no longer needed and you want to free up disk space.

When you want to remove test or temporary topics created during development.

When cleaning up after a project or service that used specific topics.

When you want to avoid confusion by removing outdated topics from the list.

When you want to reset a topic by deleting and recreating it.

Commands

This command deletes the Kafka topic named 'example-topic' from the Kafka cluster running on localhost:9092. It frees up resources used by this topic.

Terminal

kafka-topics.sh --bootstrap-server localhost:9092 --delete --topic example-topic

Expected Output

Topic example-topic is marked for deletion.

→

--delete - Marks the topic for deletion.

→

--topic - Specifies the name of the topic to delete.

→

--bootstrap-server - Specifies the Kafka server to connect to.

Lists all topics currently available in the Kafka cluster to verify that 'example-topic' has been deleted.

Terminal

kafka-topics.sh --bootstrap-server localhost:9092 --list

Expected Output

other-topic-1

other-topic-2

→

--list - Lists all topics.

→

--bootstrap-server - Specifies the Kafka server to connect to.

Key Concept

If you remember nothing else from this pattern, remember: deleting a Kafka topic marks it for removal, and you should verify its deletion by listing topics afterward.

Common Mistakes

Trying to delete a topic without enabling topic deletion in Kafka configuration.

Kafka will not delete the topic because the server setting 'delete.topic.enable' is false by default in some versions.

Ensure 'delete.topic.enable=true' is set in the Kafka broker configuration before deleting topics.

Assuming the topic is deleted immediately after running the delete command.

Kafka deletes topics asynchronously; the topic may still appear in the list for a short time.

Wait a few moments and check the topic list again to confirm deletion.

Summary

Use 'kafka-topics.sh --delete' with the topic name to mark a topic for deletion.

Verify deletion by listing topics with 'kafka-topics.sh --list'.

Make sure Kafka broker allows topic deletion by setting 'delete.topic.enable=true'.

Practice

1. What is the correct Kafka configuration to enable topic deletion?

easy

A. topic.delete.enabled=false

B. enable.topic.removal=false

C. delete.topic.enable=true

D. allow.topic.delete=no

Solution

1. Step 1: Understand Kafka topic deletion setting

   The Kafka broker must have a configuration that explicitly allows topic deletion.

2. Step 2: Identify the correct configuration key

   The correct key is delete.topic.enable and it must be set to true to allow deletion.

3. Final Answer:

   delete.topic.enable=true -> Option C

4. Quick Check:

   Topic deletion enabled = delete.topic.enable=true [OK]

Hint: Look for 'delete.topic.enable' set to true in config [OK]

Common Mistakes:

• Confusing the config key name
• Setting the value to false
• Assuming deletion works without config change

2. Which command correctly deletes a Kafka topic named my-topic?

easy

A. kafka-delete-topic.sh --topic my-topic

B. kafka-topics.sh --delete --topic my-topic

C. kafka-topics.sh --remove --topic my-topic

D. kafka-topics.sh --drop --topic my-topic

Solution

1. Step 1: Identify the correct Kafka topic deletion command

   The official Kafka command to delete a topic uses kafka-topics.sh with the --delete flag.

2. Step 2: Verify the command syntax

   The correct syntax is kafka-topics.sh --delete --topic my-topic.

3. Final Answer:

   kafka-topics.sh --delete --topic my-topic -> Option B

4. Quick Check:

   Delete topic command = kafka-topics.sh --delete --topic [OK]

Hint: Use --delete flag with kafka-topics.sh to remove topics [OK]

Common Mistakes:

• Using wrong flags like --remove or --drop
• Using a non-existent script
• Omitting the --topic option

3. After running kafka-topics.sh --delete --topic test-topic, which command confirms the topic is deleted?

medium

A. kafka-topics.sh --alter --topic test-topic

B. kafka-topics.sh --describe --topic test-topic

C. kafka-topics.sh --create --topic test-topic

D. kafka-topics.sh --list

Solution

1. Step 1: Understand how to check existing topics

   Listing topics shows all current topics in Kafka, so deleted topics won't appear.

2. Step 2: Choose the command that lists topics

   kafka-topics.sh --list lists all topics, confirming if test-topic is gone.

3. Final Answer:

   kafka-topics.sh --list -> Option D

4. Quick Check:

   List topics to confirm deletion = kafka-topics.sh --list [OK]

Hint: List topics to verify deletion, deleted ones won't show [OK]

Common Mistakes:

• Using describe which errors if topic missing
• Trying to create or alter instead of checking
• Assuming deletion is immediate without checking

4. You tried to delete a topic but it still appears in the list. What is the most likely cause?

medium

A. The broker configuration delete.topic.enable is set to false

B. You used the wrong topic name in the delete command

C. Kafka does not support topic deletion

D. The topic is currently being produced to

Solution

1. Step 1: Check broker configuration for topic deletion

   If delete.topic.enable is false, Kafka ignores delete commands and topics remain.

2. Step 2: Understand other options

   Wrong topic name causes error, Kafka supports deletion if enabled, and producing to topic doesn't block deletion.

3. Final Answer:

   The broker configuration delete.topic.enable is set to false -> Option A

4. Quick Check:

   Deletion blocked if delete.topic.enable=false [OK]

Hint: Check delete.topic.enable=true to allow deletion [OK]

Common Mistakes:

• Assuming Kafka never supports deletion
• Ignoring broker config settings
• Thinking active producers block deletion

5. You want to automate cleanup of unused topics older than 30 days. Which approach best uses Kafka's topic deletion and cleanup features?

hard

A. Enable delete.topic.enable=true and run a script that lists topics, checks their creation date, and deletes old ones

B. Set log.retention.ms to 30 days and rely on Kafka to delete topics automatically

C. Manually delete topics from Kafka config files after 30 days

D. Disable topic deletion and archive topics instead

Solution

1. Step 1: Understand Kafka's automatic cleanup capabilities

   Kafka deletes old log segments inside topics but does not delete entire topics automatically based on age.

2. Step 2: Choose an automation method for topic deletion

   Enabling delete.topic.enable=true and scripting topic listing with date checks allows controlled deletion of old topics.

3. Final Answer:

   Enable delete.topic.enable=true and run a script that lists topics, checks their creation date, and deletes old ones -> Option A

4. Quick Check:

   Automate deletion with config + script, not just retention settings [OK]

Hint: Use delete.topic.enable plus scripts for automated topic cleanup [OK]

Common Mistakes:

• Expecting Kafka to delete topics automatically by age
• Editing config files manually for topic deletion
• Disabling deletion and expecting cleanup