Kafka integration basics in Spring Boot

Introduction

Kafka integration lets your Spring Boot app send and receive messages easily. It helps different parts of your system talk to each other without waiting.

You want to send data from one app to another without delay.

You need to process messages in order and handle many at once.

You want to build a system that keeps working even if parts fail.

You want to collect logs or events from many sources in one place.

Syntax

Spring Boot

spring.kafka.bootstrap-servers=localhost:9092

@Bean
public NewTopic topic() {
    return TopicBuilder.name("myTopic").build();
}

@KafkaListener(topics = "myTopic")
public void listen(String message) {
    System.out.println("Received: " + message);
}

@Autowired
private KafkaTemplate<String, String> kafkaTemplate;

public void sendMessage(String msg) {
    kafkaTemplate.send("myTopic", msg);
}

Use spring.kafka.bootstrap-servers to set Kafka server address.

@KafkaListener marks a method to receive messages from a topic.

Examples

This listens to the 'orders' topic and prints each order message.

Spring Boot

@KafkaListener(topics = "orders")
public void receiveOrder(String order) {
    System.out.println("Order received: " + order);
}

This sends a simple text message to the 'notifications' topic.

Spring Boot

kafkaTemplate.send("notifications", "Hello World!");

This creates a topic named 'events' with 3 partitions and 1 replica.

Spring Boot

@Bean
public NewTopic createTopic() {
    return TopicBuilder.name("events").partitions(3).replicas(1).build();
}

Sample Program

This Spring Boot app creates a Kafka topic named 'demoTopic'. It sends messages to this topic and listens for messages from it. When a message arrives, it prints it to the console.

Spring Boot

package com.example.kafkademo;

import org.apache.kafka.clients.admin.NewTopic;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.annotation.EnableKafka;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.beans.factory.annotation.Autowired;

@EnableKafka
@SpringBootApplication
public class KafkaDemoApplication {

    public static void main(String[] args) {
        SpringApplication.run(KafkaDemoApplication.class, args);
    }

    @Bean
    public NewTopic demoTopic() {
        return org.springframework.kafka.config.TopicBuilder.name("demoTopic").build();
    }

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    public void sendMessage(String message) {
        kafkaTemplate.send("demoTopic", message);
    }

    @KafkaListener(topics = "demoTopic")
    public void listen(String message) {
        System.out.println("Received message: " + message);
    }
}

Important Notes

Make sure Kafka server is running on the address you set in bootstrap-servers.

Use @KafkaListener only on methods inside Spring-managed beans.

Sending and receiving messages happen asynchronously, so messages may arrive shortly after sending.

Summary

Kafka integration in Spring Boot helps apps send and receive messages easily.

Use @KafkaListener to listen to topics and KafkaTemplate to send messages.

Define topics with NewTopic beans to manage Kafka topics from your app.