What is Topic in Kafka: Definition and Usage Explained

In Apache Kafka, a topic is a category or feed name where records are stored and published. Producers send data to a topic, and consumers read data from it, enabling a simple way to organize and stream messages.

How It Works

Think of a Kafka topic like a mailbox where messages are dropped by senders and picked up by receivers. Each topic holds a stream of records, which are ordered and stored in partitions for scalability and fault tolerance.

Producers write messages to a topic, and consumers subscribe to one or more topics to receive those messages. This setup allows multiple producers and consumers to work independently but share data efficiently.

Partitions inside a topic help split the data so Kafka can handle large volumes and keep messages in order within each partition, making it easier to process data in parallel.

Example

This example shows how to create a Kafka topic and produce a message to it using the Kafka command-line tools.

bash

kafka-topics.sh --create --topic my-topic --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1

kafka-console-producer.sh --topic my-topic --bootstrap-server localhost:9092
Hello Kafka!

Output

Created topic my-topic.
Hello Kafka!

When to Use

Use Kafka topics when you need to stream data between different parts of your system in real time. For example, you can use topics to collect user activity logs, send sensor data from devices, or integrate microservices.

Topics help organize data streams by category, making it easier to scale and manage data flow. They are ideal for event-driven architectures, real-time analytics, and building reliable data pipelines.

Key Points

• A topic is a named stream of messages in Kafka.
• Messages are stored in partitions within a topic for scalability.
• Producers write to topics; consumers read from topics.
• Topics enable real-time data streaming and decoupling of systems.

Key Takeaways

A Kafka topic is a named channel where messages are published and consumed.

Topics are divided into partitions to allow parallel processing and scalability.

Use topics to organize and stream data efficiently between producers and consumers.

Kafka topics support real-time data pipelines and event-driven systems.