How to Configure Kafka Producer in Spring Boot

To configure a Kafka producer in Spring Boot, define producer properties in application.properties or application.yml and create a KafkaTemplate bean. Use this template to send messages to Kafka topics easily.

Syntax

Kafka producer configuration in Spring Boot involves setting properties like bootstrap.servers, key.serializer, and value.serializer. Then, create a ProducerFactory and a KafkaTemplate bean to send messages.

Key parts explained:

• bootstrap.servers: Kafka broker addresses.
• key.serializer: Class to serialize message keys.
• value.serializer: Class to serialize message values.
• ProducerFactory: Creates Kafka producers with config.
• KafkaTemplate: Sends messages to Kafka topics.

java

spring.kafka.producer.bootstrap-servers=localhost:9092
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer

@Bean
public ProducerFactory<String, String> producerFactory() {
    Map<String, Object> configProps = new HashMap<>();
    configProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
    configProps.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
    configProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
    return new DefaultKafkaProducerFactory<>(configProps);
}

@Bean
public KafkaTemplate<String, String> kafkaTemplate() {
    return new KafkaTemplate<>(producerFactory());
}

Example

This example shows a Spring Boot application that configures a Kafka producer and sends a simple message to a topic named test-topic.

java

import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.common.serialization.StringSerializer;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.kafka.core.DefaultKafkaProducerFactory;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.core.ProducerFactory;

import java.util.HashMap;
import java.util.Map;

@SpringBootApplication
public class KafkaProducerApplication {

    public static void main(String[] args) {
        var context = SpringApplication.run(KafkaProducerApplication.class, args);
        KafkaTemplate<String, String> kafkaTemplate = context.getBean(KafkaTemplate.class);
        kafkaTemplate.send("test-topic", "Hello Kafka from Spring Boot!");
        System.out.println("Message sent to Kafka topic 'test-topic'");
    }

    @Bean
    public ProducerFactory<String, String> producerFactory() {
        Map<String, Object> configProps = new HashMap<>();
        configProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
        configProps.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        configProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        return new DefaultKafkaProducerFactory<>(configProps);
    }

    @Bean
    public KafkaTemplate<String, String> kafkaTemplate() {
        return new KafkaTemplate<>(producerFactory());
    }
}

Output

Message sent to Kafka topic 'test-topic'

Common Pitfalls

Common mistakes when configuring Kafka producer in Spring include:

• Not setting bootstrap.servers correctly, causing connection failures.
• Using wrong serializer classes, leading to serialization errors.
• Forgetting to create KafkaTemplate bean, so messages cannot be sent.
• Not handling exceptions when sending messages.

Always verify Kafka broker is running and reachable.

java

/* Wrong serializer example - causes runtime error */
configProps.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, org.apache.kafka.common.serialization.ByteArraySerializer.class);

/* Correct serializer example */
configProps.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, org.apache.kafka.common.serialization.StringSerializer.class);

Quick Reference

Property	Description	Example Value
spring.kafka.producer.bootstrap-servers	Kafka broker addresses	localhost:9092
spring.kafka.producer.key-serializer	Serializer for message keys	org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer	Serializer for message values	org.apache.kafka.common.serialization.StringSerializer
ProducerFactory	Creates Kafka producer instances	DefaultKafkaProducerFactory with config map
KafkaTemplate	Sends messages to Kafka topics	KafkaTemplate bean

Key Takeaways

Set correct Kafka broker addresses in bootstrap.servers property.

Use StringSerializer for key and value serializers for text messages.

Create ProducerFactory and KafkaTemplate beans to send messages.

Always verify Kafka broker is running before sending messages.

Handle exceptions when sending messages to avoid silent failures.