Kafka CLI tools overview - Mini Project: Build & Apply

Kafka CLI tools overview

📖 Scenario: You are working with Apache Kafka, a system that helps send messages between different parts of a program. To manage Kafka, you use command-line tools that let you create topics, send messages, and check the system's status.

🎯 Goal: Learn how to use basic Kafka command-line tools to create a topic, list topics, send messages, and read messages from a topic.

📋 What You'll Learn

Use the kafka-topics.sh tool to create and list topics

Use the kafka-console-producer.sh tool to send messages

Use the kafka-console-consumer.sh tool to read messages

Use exact commands and options as specified

💡 Why This Matters

🌍 Real World

Kafka CLI tools are used by developers and system administrators to manage Kafka clusters, topics, and messages directly from the command line.

💼 Career

Knowing Kafka CLI commands is essential for roles involving data streaming, real-time analytics, and distributed system management.

1

Create a Kafka topic

Use the kafka-topics.sh command with --create to create a topic named test-topic with 1 partition and replication factor 1. Include --bootstrap-server localhost:9092 in the command.

Kafka

# Type the command to create the topic test-topic
# Your code here

Need a hint?

Remember to specify the topic name exactly as test-topic and use --bootstrap-server localhost:9092 to connect.

2

List Kafka topics

Use the kafka-topics.sh command with --list and --bootstrap-server localhost:9092 to list all topics.

Kafka

kafka-topics.sh --create --topic test-topic --partitions 1 --replication-factor 1 --bootstrap-server localhost:9092
# Type the command to list all topics
# Your code here

Need a hint?

Use --list to see all topics available on the server.

3

Send messages to the topic

Use the kafka-console-producer.sh command with --topic test-topic and --bootstrap-server localhost:9092 to send messages. Type the message Hello Kafka and then stop the producer.

Kafka

kafka-topics.sh --create --topic test-topic --partitions 1 --replication-factor 1 --bootstrap-server localhost:9092
kafka-topics.sh --list --bootstrap-server localhost:9092
# Type the command to start the producer for test-topic
# Then type 'Hello Kafka' as the message
# Your code here

Need a hint?

Run the producer command, then type the message Hello Kafka and press Enter.

4

Read messages from the topic

Use the kafka-console-consumer.sh command with --topic test-topic, --from-beginning, and --bootstrap-server localhost:9092 to read all messages from the start. The output should show Hello Kafka.

Kafka

kafka-topics.sh --create --topic test-topic --partitions 1 --replication-factor 1 --bootstrap-server localhost:9092
kafka-topics.sh --list --bootstrap-server localhost:9092
kafka-console-producer.sh --topic test-topic --bootstrap-server localhost:9092
Hello Kafka
# Type the command to consume messages from test-topic from the beginning
# Your code here

Need a hint?

Use --from-beginning to read all messages from the start of the topic.