Kafka integration basics in Spring Boot - Mini Project: Build & Apply

Kafka integration basics

📖 Scenario: You are building a simple Spring Boot application that sends and receives messages using Kafka. This is like sending letters through a post office and then reading them when they arrive.

🎯 Goal: Create a Spring Boot application that sets up a Kafka topic, sends a message to it, and listens for messages from that topic.

📋 What You'll Learn

Create a Kafka topic configuration bean

Create a Kafka producer bean to send messages

Create a Kafka consumer bean to listen for messages

Send a test message to the Kafka topic

💡 Why This Matters

🌍 Real World

Kafka is used in many applications to send messages between different parts of a system reliably and quickly, like sending notifications or processing orders.

💼 Career

Understanding Kafka integration with Spring Boot is valuable for backend developers working on scalable, event-driven systems.

1

Create Kafka topic configuration

Create a Spring bean method called topic that returns a NewTopic object with the name "myTopic", 1 partition, and 1 replica.

Spring Boot

# Create a Spring bean method called topic that returns a NewTopic with name "myTopic", 1 partition, and 1 replica
# Your code here

Hint

Use @Bean annotation and return new NewTopic("myTopic", 1, (short) 1).

2

Create Kafka producer bean

Add a Spring bean method called kafkaTemplate that returns a KafkaTemplate<String, String> using a ProducerFactory<String, String> parameter named producerFactory.

Spring Boot

import org.apache.kafka.clients.admin.NewTopic;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.core.ProducerFactory;

@Configuration
public class KafkaTopicConfig {

    @Bean
    public NewTopic topic() {
        return new NewTopic("myTopic", 1, (short) 1);
    }

    // Create kafkaTemplate bean here
    // Your code here
}

Hint

Define a @Bean method named kafkaTemplate that takes ProducerFactory<String, String> and returns new KafkaTemplate<>(producerFactory).

3

Create Kafka consumer listener

Create a Spring component class called KafkaConsumer with a method listen annotated with @KafkaListener(topics = "myTopic") that takes a String message parameter.

Spring Boot

import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;

@Component
public class KafkaConsumer {

    // Create listen method with @KafkaListener for topic "myTopic"
    // Your code here
}

Hint

Use @Component on the class and @KafkaListener(topics = "myTopic") on the method that takes a String message.

4

Send a test message to Kafka topic

In a Spring @Component class called KafkaSender, create a method sendMessage that takes a KafkaTemplate<String, String> named kafkaTemplate and sends the message "Hello Kafka" to the topic "myTopic" using kafkaTemplate.send("myTopic", "Hello Kafka").

Spring Boot

import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Component;

@Component
public class KafkaSender {

    // Create sendMessage method that sends "Hello Kafka" to "myTopic"
    // Your code here
}

Hint

Define a @Component class with a method sendMessage that calls kafkaTemplate.send("myTopic", "Hello Kafka").

Practice

1. What is the primary purpose of @KafkaListener in a Spring Boot application?

easy

A. To listen and consume messages from a Kafka topic

B. To send messages to a Kafka topic

C. To configure Kafka broker settings

D. To create new Kafka topics automatically

Solution

1. Step 1: Understand the role of @KafkaListener

   This annotation marks a method to receive messages from a Kafka topic automatically.

2. Step 2: Differentiate sending and listening

   Sending messages is done with KafkaTemplate, not @KafkaListener.

3. Final Answer:

   To listen and consume messages from a Kafka topic -> Option A

4. Quick Check:

   @KafkaListener listens to topics [OK]

Hint: Remember: Listener receives, Template sends [OK]

Common Mistakes:

• Confusing @KafkaListener with KafkaTemplate
• Thinking @KafkaListener creates topics
• Assuming it configures Kafka brokers

2. Which of the following is the correct way to send a message to a Kafka topic using Spring Boot?

easy

A. new KafkaProducer().send("topicName", "message")

B. @KafkaListener(topic = "topicName")

C. KafkaTemplate.listen("topicName", "message")

D. kafkaTemplate.send("topicName", "message")

Solution

1. Step 1: Identify the sending method in Spring Boot

   Spring Boot uses KafkaTemplate with the send() method to send messages.

2. Step 2: Check syntax correctness

   kafkaTemplate.send("topicName", "message") is the correct syntax to send a message.

3. Final Answer:

   kafkaTemplate.send("topicName", "message") -> Option D

4. Quick Check:

   Sending uses KafkaTemplate.send() [OK]

Hint: Send messages with kafkaTemplate.send() [OK]

Common Mistakes:

• Using @KafkaListener to send messages
• Calling listen() instead of send()
• Instantiating KafkaProducer manually in Spring Boot

3. Given this Spring Boot Kafka listener method:

@KafkaListener(topics = "myTopic")
public void listen(String message) {
    System.out.println("Received: " + message);
}

What will be printed if a message "Hello Kafka" is sent to myTopic?

medium

A. Received: Hello Kafka

B. Hello Kafka

C. Received message: Hello Kafka

D. Error: No message received

Solution

1. Step 1: Analyze the listener method output

   The method prints "Received: " concatenated with the message string.

2. Step 2: Substitute the message value

   Message is "Hello Kafka", so output is "Received: Hello Kafka".

3. Final Answer:

   Received: Hello Kafka -> Option A

4. Quick Check:

   Prints "Received: " + message [OK]

Hint: Check print statement format carefully [OK]

Common Mistakes:

• Ignoring the prefix in the print statement
• Assuming only the message prints
• Confusing topic name with output

4. Consider this Spring Boot Kafka configuration snippet:

@Bean
public NewTopic topic() {
    return new NewTopic("testTopic", 1, (short) 1);
}

What is the likely error if the application fails to create the topic?

medium

A. Incorrect topic name format

B. Missing Kafka broker connection settings

C. NewTopic requires 3 parameters but only 2 provided

D. NewTopic bean must be annotated with @KafkaListener

Solution

1. Step 1: Understand NewTopic bean usage

   NewTopic defines topic name, partitions, and replication factor correctly here.

2. Step 2: Identify common cause for topic creation failure

   Failure usually happens if Kafka broker connection settings are missing or incorrect.

3. Final Answer:

   Missing Kafka broker connection settings -> Option B

4. Quick Check:

   Broker settings needed for topic creation [OK]

Hint: Check Kafka broker config if topic creation fails [OK]

Common Mistakes:

• Thinking @KafkaListener is needed on NewTopic bean
• Assuming topic name format is invalid
• Miscounting NewTopic constructor parameters

5. You want to create a Spring Boot Kafka consumer that listens to two topics: topicA and topicB. Which of the following @KafkaListener annotations correctly achieves this?

hard

A. @KafkaListener(topics = "topicA, topicB")

B. @KafkaListener(topic = "topicA,topicB")

C. @KafkaListener(topics = {"topicA", "topicB"})

D. @KafkaListener(topics = ["topicA", "topicB"])

Solution

1. Step 1: Recall correct syntax for multiple topics

   Spring Boot expects an array of strings for multiple topics: topics = {"topicA", "topicB"}.

2. Step 2: Evaluate other options

   @KafkaListener(topic = "topicA,topicB") uses wrong attribute name and comma string, C uses string with comma but not array, D uses invalid syntax with square brackets.

3. Final Answer:

   @KafkaListener(topics = {"topicA", "topicB"}) -> Option C

4. Quick Check:

   Multiple topics require array syntax [OK]

Hint: Use curly braces for multiple topics in @KafkaListener [OK]

Common Mistakes:

• Using comma-separated string instead of array
• Wrong attribute name 'topic' instead of 'topics'
• Using square brackets instead of curly braces