What is Producer in Kafka: Definition and Usage

A producer in Kafka is a client application that sends data (messages) to Kafka topics. It acts like a sender that publishes records to Kafka so other applications can consume them.

How It Works

Think of Kafka as a large message post office. The producer is like a person who writes letters and drops them into the post office boxes (called topics). Each letter is a message that contains some data.

The producer decides which topic to send the message to and can also choose which partition inside the topic to use. Kafka then stores these messages reliably so other applications, called consumers, can read them later.

This system allows many producers to send messages independently and quickly, while Kafka manages the storage and delivery to consumers.

Example

This example shows a simple Kafka producer in Java that sends a message to a topic named my-topic.

java

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.clients.producer.RecordMetadata;
import java.util.Properties;

public class SimpleProducer {
    public static void main(String[] args) throws Exception {
        Properties props = new Properties();
        props.put("bootstrap.servers", "localhost:9092");
        props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

        try (KafkaProducer<String, String> producer = new KafkaProducer<>(props)) {
            ProducerRecord<String, String> record = new ProducerRecord<>("my-topic", "key1", "Hello Kafka!");
            RecordMetadata metadata = producer.send(record).get();
            System.out.println("Message sent to topic " + metadata.topic() + " partition " + metadata.partition() + " offset " + metadata.offset());
        }
    }
}

Output

Message sent to topic my-topic partition 0 offset 15

When to Use

Use a Kafka producer whenever you need to send data streams or events to Kafka for processing or storage. Common use cases include:

• Logging systems sending logs to Kafka for analysis.
• Web applications sending user activity events.
• IoT devices sending sensor data.
• Microservices communicating asynchronously by producing messages.

Producers help decouple systems by allowing data to flow reliably and asynchronously into Kafka topics.

Key Points

• A producer sends messages to Kafka topics.
• It controls which topic and partition to send data to.
• Producers use serializers to convert data into bytes.
• Kafka producers are asynchronous and scalable.
• They enable real-time data streaming and event-driven architectures.

Key Takeaways

A Kafka producer sends data messages to Kafka topics for other applications to consume.

Producers decide the topic and partition for each message to organize data flow.

They use serializers to convert data into a format Kafka can store and transmit.

Kafka producers enable scalable, asynchronous, and real-time data streaming.

Use producers to send logs, events, sensor data, or messages between services.