How to Use Kafka Producer API: Simple Guide with Example

To use the KafkaProducer API, create a producer instance with configuration properties, then call send() to publish messages to a Kafka topic asynchronously. Always close the producer after use to free resources.

Syntax

The basic syntax to use Kafka Producer API involves creating a KafkaProducer object with configuration properties, then sending messages using the send() method. Finally, close the producer to release resources.

• Properties: Configure bootstrap servers, key and value serializers.
• KafkaProducer: The client object to send messages.
• ProducerRecord: Represents the message with topic, key, and value.
• send(): Sends the message asynchronously.
• close(): Closes the producer cleanly.

java

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

KafkaProducer<String, String> producer = new KafkaProducer<>(props);

ProducerRecord<String, String> record = new ProducerRecord<>("topic-name", "key", "value");

producer.send(record);

producer.close();

Example

This example shows how to send a simple message to a Kafka topic named test-topic. It demonstrates setting up the producer, sending a message, and closing the producer.

java

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import java.util.Properties;

public class SimpleProducer {
    public static void main(String[] args) {
        Properties props = new Properties();
        props.put("bootstrap.servers", "localhost:9092");
        props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

        KafkaProducer<String, String> producer = new KafkaProducer<>(props);

        ProducerRecord<String, String> record = new ProducerRecord<>("test-topic", "myKey", "Hello Kafka");

        producer.send(record);

        producer.close();
        System.out.println("Message sent successfully");
    }
}

Output

Message sent successfully

Common Pitfalls

Common mistakes when using Kafka Producer API include:

• Not setting the correct bootstrap.servers address, causing connection failures.
• Forgetting to specify serializers for key and value, leading to serialization errors.
• Not closing the producer, which can cause resource leaks.
• Assuming send() is synchronous; it is asynchronous and may require callbacks or get() for confirmation.

java

/* Wrong: Missing serializers */
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
KafkaProducer<String, String> producer = new KafkaProducer<>(props); // This will fail

/* Right: Include serializers */
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
KafkaProducer<String, String> producerCorrect = new KafkaProducer<>(props);

Quick Reference

Remember these key points when using Kafka Producer API:

• Always configure bootstrap.servers and serializers.
• Use ProducerRecord to specify topic, key, and value.
• send() is asynchronous; use callbacks or get() if you need confirmation.
• Close the producer with close() to free resources.

Key Takeaways

Configure bootstrap servers and serializers before creating KafkaProducer.

Use ProducerRecord to send messages specifying topic, key, and value.

send() is asynchronous; handle delivery confirmation if needed.

Always close the producer to avoid resource leaks.

Check connection and serialization settings to avoid common errors.