What Is Broker in Kafka: Simple Explanation and Example
broker is a server that stores and manages message data. It receives messages from producers, stores them safely, and sends them to consumers when requested.How It Works
Think of a Kafka broker as a post office in a city. Producers send letters (messages) to the post office, which then sorts and stores them safely. When someone wants to read a letter, the post office delivers it to them.
In Kafka, the broker handles this by storing messages in topics, which are like mailboxes. Each broker can manage many topics and partitions, ensuring messages are kept in order and available for consumers to read at any time.
Multiple brokers work together in a Kafka cluster to share the load and provide fault tolerance. If one broker fails, others continue to serve messages, keeping the system reliable.
Example
This example shows how to list brokers in a Kafka cluster using the Kafka command-line tool.
bin/kafka-broker-api-versions.sh --bootstrap-server localhost:9092When to Use
You use Kafka brokers whenever you need a reliable system to send, store, and receive streams of data in real time. Brokers are essential for building event-driven applications, log aggregation, or real-time analytics.
For example, an online store can use Kafka brokers to handle orders as messages, ensuring every order is recorded and processed even if some servers go down.
Also, brokers help scale your system by distributing data across multiple servers, so your application can handle more users and data without slowing down.
Key Points
- A Kafka broker is a server that stores and manages messages.
- It receives data from producers and sends it to consumers.
- Brokers work together in clusters for reliability and scalability.
- They organize messages in topics and partitions.
- Using brokers helps build real-time, fault-tolerant data systems.