Subscribing to topics in Kafka - Commands & Configuration

Introduction

When you want to receive messages from a Kafka topic, you subscribe to it. This lets your application listen and react to new data as it arrives.

When you want to process user activity logs in real-time.

When you need to update a dashboard with live data from sensors.

When you want to trigger actions based on events like orders placed.

When you want to build a chat app that shows messages instantly.

When you want to collect and analyze streaming data continuously.

Commands

This command starts a Kafka consumer that connects to the Kafka server at localhost on port 9092. It subscribes to the topic named 'example-topic' and reads all messages from the start of the topic.

Terminal

kafka-console-consumer --bootstrap-server localhost:9092 --topic example-topic --from-beginning

Expected Output

This will print all messages from 'example-topic' to the terminal as they arrive, starting from the oldest message.

For example:

Hello Kafka

New event received

User logged in

→

--bootstrap-server - Specifies the Kafka server address to connect to.

→

--topic - Specifies the topic to subscribe to.

→

--from-beginning - Reads all messages from the start of the topic instead of only new messages.

This command subscribes to 'example-topic' but only shows new messages arriving after the command starts, ignoring old messages.

Terminal

kafka-console-consumer --bootstrap-server localhost:9092 --topic example-topic

Expected Output

Only new messages sent to 'example-topic' after this command runs will appear here.

For example:

New order placed

Payment processed

→

--bootstrap-server - Specifies the Kafka server address.

→

--topic - Specifies the topic to listen to.

Key Concept

If you remember nothing else from this pattern, remember: subscribing to a Kafka topic lets your app receive messages as they come in, either from the start or only new ones.

Common Mistakes

Not specifying the correct Kafka server address with --bootstrap-server.

The consumer cannot connect to Kafka and will fail to receive messages.

Always provide the correct Kafka server address and port using --bootstrap-server.

Forgetting to specify the topic name with --topic.

Kafka consumer will not know which topic to listen to and will error out.

Always include the --topic flag followed by the exact topic name.

Expecting old messages without using --from-beginning.

By default, the consumer only shows new messages after it starts.

Add --from-beginning if you want to read all existing messages from the topic.

Summary

Use kafka-console-consumer with --bootstrap-server and --topic to subscribe to a Kafka topic.

Add --from-beginning to read all messages from the start, otherwise only new messages appear.

Check your Kafka server address and topic name carefully to avoid connection errors.