Topic deletion and cleanup in Kafka - Mini Project: Build & Apply

Kafka Topic Deletion and Cleanup

📖 Scenario: You are managing a Kafka cluster that stores messages for different applications. Sometimes, old topics are no longer needed and should be deleted to free up space and keep the system clean.

🎯 Goal: You will learn how to delete a Kafka topic safely and verify that the topic has been removed from the cluster.

📋 What You'll Learn

Kafka cluster is running locally

Kafka command-line tools are available

Basic knowledge of terminal commands

💡 Why This Matters

🌍 Real World

Kafka topics store streams of data for applications. Removing unused topics helps keep the system clean and saves storage.

💼 Career

Kafka administrators and DevOps engineers often need to manage topics, including deleting old ones to maintain cluster health.

1

Create a Kafka topic named old_topic

Use the Kafka command-line tool to create a topic called old_topic with 1 partition and a replication factor of 1.

Kafka

# Your code here

Hint

Use kafka-topics.sh --create with the right flags to create the topic.

2

Enable topic deletion in Kafka configuration

Set the Kafka broker configuration property delete.topic.enable to true to allow topic deletion.

Kafka

kafka-topics.sh --create --topic old_topic --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1
# Your code here

Hint

Find the server.properties file and add or update delete.topic.enable=true. Restart Kafka after.

3

Delete the Kafka topic named old_topic

Use the Kafka command-line tool to delete the topic called old_topic.

Kafka

kafka-topics.sh --create --topic old_topic --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1
# Edit the server.properties file and set delete.topic.enable=true
# Then restart the Kafka broker to apply the change
# Your code here

Hint

Use kafka-topics.sh --delete --topic old_topic to remove the topic.

4

Verify that the topic old_topic is deleted

List all Kafka topics and confirm that old_topic no longer appears.

Kafka

kafka-topics.sh --create --topic old_topic --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1
# Edit the server.properties file and set delete.topic.enable=true
# Then restart the Kafka broker to apply the change
kafka-topics.sh --delete --topic old_topic --bootstrap-server localhost:9092
# Verify deletion and print confirmation here

Hint

Use kafka-topics.sh --list and check that old_topic is not shown.

Practice

1. What is the correct Kafka configuration to enable topic deletion?

easy

A. topic.delete.enabled=false

B. enable.topic.removal=false

C. delete.topic.enable=true

D. allow.topic.delete=no

Solution

1. Step 1: Understand Kafka topic deletion setting

   The Kafka broker must have a configuration that explicitly allows topic deletion.

2. Step 2: Identify the correct configuration key

   The correct key is delete.topic.enable and it must be set to true to allow deletion.

3. Final Answer:

   delete.topic.enable=true -> Option C

4. Quick Check:

   Topic deletion enabled = delete.topic.enable=true [OK]

Hint: Look for 'delete.topic.enable' set to true in config [OK]

Common Mistakes:

• Confusing the config key name
• Setting the value to false
• Assuming deletion works without config change

2. Which command correctly deletes a Kafka topic named my-topic?

easy

A. kafka-delete-topic.sh --topic my-topic

B. kafka-topics.sh --delete --topic my-topic

C. kafka-topics.sh --remove --topic my-topic

D. kafka-topics.sh --drop --topic my-topic

Solution

1. Step 1: Identify the correct Kafka topic deletion command

   The official Kafka command to delete a topic uses kafka-topics.sh with the --delete flag.

2. Step 2: Verify the command syntax

   The correct syntax is kafka-topics.sh --delete --topic my-topic.

3. Final Answer:

   kafka-topics.sh --delete --topic my-topic -> Option B

4. Quick Check:

   Delete topic command = kafka-topics.sh --delete --topic [OK]

Hint: Use --delete flag with kafka-topics.sh to remove topics [OK]

Common Mistakes:

• Using wrong flags like --remove or --drop
• Using a non-existent script
• Omitting the --topic option

3. After running kafka-topics.sh --delete --topic test-topic, which command confirms the topic is deleted?

medium

A. kafka-topics.sh --alter --topic test-topic

B. kafka-topics.sh --describe --topic test-topic

C. kafka-topics.sh --create --topic test-topic

D. kafka-topics.sh --list

Solution

1. Step 1: Understand how to check existing topics

   Listing topics shows all current topics in Kafka, so deleted topics won't appear.

2. Step 2: Choose the command that lists topics

   kafka-topics.sh --list lists all topics, confirming if test-topic is gone.

3. Final Answer:

   kafka-topics.sh --list -> Option D

4. Quick Check:

   List topics to confirm deletion = kafka-topics.sh --list [OK]

Hint: List topics to verify deletion, deleted ones won't show [OK]

Common Mistakes:

• Using describe which errors if topic missing
• Trying to create or alter instead of checking
• Assuming deletion is immediate without checking

4. You tried to delete a topic but it still appears in the list. What is the most likely cause?

medium

A. The broker configuration delete.topic.enable is set to false

B. You used the wrong topic name in the delete command

C. Kafka does not support topic deletion

D. The topic is currently being produced to

Solution

1. Step 1: Check broker configuration for topic deletion

   If delete.topic.enable is false, Kafka ignores delete commands and topics remain.

2. Step 2: Understand other options

   Wrong topic name causes error, Kafka supports deletion if enabled, and producing to topic doesn't block deletion.

3. Final Answer:

   The broker configuration delete.topic.enable is set to false -> Option A

4. Quick Check:

   Deletion blocked if delete.topic.enable=false [OK]

Hint: Check delete.topic.enable=true to allow deletion [OK]

Common Mistakes:

• Assuming Kafka never supports deletion
• Ignoring broker config settings
• Thinking active producers block deletion

5. You want to automate cleanup of unused topics older than 30 days. Which approach best uses Kafka's topic deletion and cleanup features?

hard

A. Enable delete.topic.enable=true and run a script that lists topics, checks their creation date, and deletes old ones

B. Set log.retention.ms to 30 days and rely on Kafka to delete topics automatically

C. Manually delete topics from Kafka config files after 30 days

D. Disable topic deletion and archive topics instead

Solution

1. Step 1: Understand Kafka's automatic cleanup capabilities

   Kafka deletes old log segments inside topics but does not delete entire topics automatically based on age.

2. Step 2: Choose an automation method for topic deletion

   Enabling delete.topic.enable=true and scripting topic listing with date checks allows controlled deletion of old topics.

3. Final Answer:

   Enable delete.topic.enable=true and run a script that lists topics, checks their creation date, and deletes old ones -> Option A

4. Quick Check:

   Automate deletion with config + script, not just retention settings [OK]

Hint: Use delete.topic.enable plus scripts for automated topic cleanup [OK]

Common Mistakes:

• Expecting Kafka to delete topics automatically by age
• Editing config files manually for topic deletion
• Disabling deletion and expecting cleanup