Kafka integration basics in Spring Boot - Step-by-Step Execution

Concept Flow - Kafka integration basics

Start Spring Boot App

↓

Configure Kafka Producer

↓

Send Message to Topic

↓

Kafka Broker Receives Message

↓

Configure Kafka Consumer

↓

Consumer Listens to Topic

↓

Process Received Message

↓

End

This flow shows how a Spring Boot app configures Kafka producer and consumer, sends a message, and processes it.

Execution Sample

Spring Boot

1. @SpringBootApplication
2. @KafkaListener(topics = "test-topic")
3. public void listen(String msg) { System.out.println(msg); }
4. kafkaTemplate.send("test-topic", "Hello Kafka");

This code sends a message to Kafka and listens to the same topic to print received messages.

Execution Table

Step	Action	Component	Message/Value	Result
1	Start Spring Boot Application	Application	-	App initialized with Kafka configs
2	Configure Kafka Producer	Producer	-	Producer ready to send messages
3	Configure Kafka Consumer with @KafkaListener	Consumer	topic=test-topic	Consumer subscribed to topic
4	Send message 'Hello Kafka' to 'test-topic'	Producer	Hello Kafka	Message sent to Kafka broker
5	Kafka broker receives message	Kafka Broker	Hello Kafka	Message stored in topic partition
6	Consumer listens and receives message	Consumer	Hello Kafka	Message received by consumer
7	Consumer processes message	Consumer	Hello Kafka	Message printed to console
8	End	-	-	Message flow complete

💡 Message sent and received successfully; flow ends after processing.

Variable Tracker

Variable	Start	After Step 4	After Step 6	Final
message	null	"Hello Kafka" (sent)	"Hello Kafka" (received)	null (processed)
producerReady	false	true	true	true
consumerReady	false	true	true	true

Key Moments - 3 Insights

Why does the consumer receive the message after it is sent?

Because the consumer is subscribed to the same topic and listens continuously, as shown in execution_table step 6.

What happens if the producer sends a message before the consumer is ready?

The message stays in the Kafka broker until the consumer subscribes and receives it, ensuring no message loss.

Why do we use @KafkaListener annotation?

It tells Spring Boot which method should listen to messages from a specific Kafka topic, as seen in step 3.

Visual Quiz - 3 Questions

Look at the execution table, at which step does the Kafka broker store the message?

A Step 6

B Step 4

C Step 5

D Step 3

💡 Hint

Check the 'Kafka Broker' component action in the execution_table rows.

According to variable_tracker, what is the value of 'message' after step 6?

A "Hello Kafka" (received)

B null

C "Hello Kafka" (sent)

D undefined

💡 Hint

Look at the 'message' row under 'After Step 6' column in variable_tracker.

If the consumer was not configured, what would happen to the message?

A Message would be lost immediately

B Message would remain in Kafka broker until consumed

C Message would be printed to console anyway

D Application would crash

💡 Hint

Refer to key_moments explanation about message storage in Kafka broker.

Concept Snapshot

Kafka integration in Spring Boot:
- Configure producer and consumer beans
- Use kafkaTemplate.send(topic, message) to send
- Use @KafkaListener to receive messages
- Messages go through Kafka broker
- Consumer processes messages asynchronously
- Ensures reliable message delivery

Full Transcript

This visual execution trace shows how a Spring Boot application integrates with Kafka. The app starts and configures a Kafka producer and consumer. The producer sends a message 'Hello Kafka' to a topic named 'test-topic'. The Kafka broker receives and stores this message. The consumer, subscribed to the same topic via @KafkaListener, receives the message and processes it by printing to the console. Variables like 'message', 'producerReady', and 'consumerReady' track the state changes during execution. Key moments clarify why the consumer receives messages and the role of the Kafka broker. The quiz tests understanding of message flow and state changes. This trace helps beginners see step-by-step how Kafka messaging works in Spring Boot.

Practice

1. What is the primary purpose of @KafkaListener in a Spring Boot application?

easy

A. To listen and consume messages from a Kafka topic

B. To send messages to a Kafka topic

C. To configure Kafka broker settings

D. To create new Kafka topics automatically

Solution

1. Step 1: Understand the role of @KafkaListener

   This annotation marks a method to receive messages from a Kafka topic automatically.

2. Step 2: Differentiate sending and listening

   Sending messages is done with KafkaTemplate, not @KafkaListener.

3. Final Answer:

   To listen and consume messages from a Kafka topic -> Option A

4. Quick Check:

   @KafkaListener listens to topics [OK]

Hint: Remember: Listener receives, Template sends [OK]

Common Mistakes:

• Confusing @KafkaListener with KafkaTemplate
• Thinking @KafkaListener creates topics
• Assuming it configures Kafka brokers

2. Which of the following is the correct way to send a message to a Kafka topic using Spring Boot?

easy

A. new KafkaProducer().send("topicName", "message")

B. @KafkaListener(topic = "topicName")

C. KafkaTemplate.listen("topicName", "message")

D. kafkaTemplate.send("topicName", "message")

Solution

1. Step 1: Identify the sending method in Spring Boot

   Spring Boot uses KafkaTemplate with the send() method to send messages.

2. Step 2: Check syntax correctness

   kafkaTemplate.send("topicName", "message") is the correct syntax to send a message.

3. Final Answer:

   kafkaTemplate.send("topicName", "message") -> Option D

4. Quick Check:

   Sending uses KafkaTemplate.send() [OK]

Hint: Send messages with kafkaTemplate.send() [OK]

Common Mistakes:

• Using @KafkaListener to send messages
• Calling listen() instead of send()
• Instantiating KafkaProducer manually in Spring Boot

3. Given this Spring Boot Kafka listener method:

@KafkaListener(topics = "myTopic")
public void listen(String message) {
    System.out.println("Received: " + message);
}

What will be printed if a message "Hello Kafka" is sent to myTopic?

medium

A. Received: Hello Kafka

B. Hello Kafka

C. Received message: Hello Kafka

D. Error: No message received

Solution

1. Step 1: Analyze the listener method output

   The method prints "Received: " concatenated with the message string.

2. Step 2: Substitute the message value

   Message is "Hello Kafka", so output is "Received: Hello Kafka".

3. Final Answer:

   Received: Hello Kafka -> Option A

4. Quick Check:

   Prints "Received: " + message [OK]

Hint: Check print statement format carefully [OK]

Common Mistakes:

• Ignoring the prefix in the print statement
• Assuming only the message prints
• Confusing topic name with output

4. Consider this Spring Boot Kafka configuration snippet:

@Bean
public NewTopic topic() {
    return new NewTopic("testTopic", 1, (short) 1);
}

What is the likely error if the application fails to create the topic?

medium

A. Incorrect topic name format

B. Missing Kafka broker connection settings

C. NewTopic requires 3 parameters but only 2 provided

D. NewTopic bean must be annotated with @KafkaListener

Solution

1. Step 1: Understand NewTopic bean usage

   NewTopic defines topic name, partitions, and replication factor correctly here.

2. Step 2: Identify common cause for topic creation failure

   Failure usually happens if Kafka broker connection settings are missing or incorrect.

3. Final Answer:

   Missing Kafka broker connection settings -> Option B

4. Quick Check:

   Broker settings needed for topic creation [OK]

Hint: Check Kafka broker config if topic creation fails [OK]

Common Mistakes:

• Thinking @KafkaListener is needed on NewTopic bean
• Assuming topic name format is invalid
• Miscounting NewTopic constructor parameters

5. You want to create a Spring Boot Kafka consumer that listens to two topics: topicA and topicB. Which of the following @KafkaListener annotations correctly achieves this?

hard

A. @KafkaListener(topics = "topicA, topicB")

B. @KafkaListener(topic = "topicA,topicB")

C. @KafkaListener(topics = {"topicA", "topicB"})

D. @KafkaListener(topics = ["topicA", "topicB"])

Solution

1. Step 1: Recall correct syntax for multiple topics

   Spring Boot expects an array of strings for multiple topics: topics = {"topicA", "topicB"}.

2. Step 2: Evaluate other options

   @KafkaListener(topic = "topicA,topicB") uses wrong attribute name and comma string, C uses string with comma but not array, D uses invalid syntax with square brackets.

3. Final Answer:

   @KafkaListener(topics = {"topicA", "topicB"}) -> Option C

4. Quick Check:

   Multiple topics require array syntax [OK]

Hint: Use curly braces for multiple topics in @KafkaListener [OK]

Common Mistakes:

• Using comma-separated string instead of array
• Wrong attribute name 'topic' instead of 'topics'
• Using square brackets instead of curly braces