Java consumer client in Kafka - Mini Project: Build & Apply

Java Kafka Consumer Client

📖 Scenario: You are building a simple Java application that reads messages from a Kafka topic. Kafka is a system used to send and receive messages between different parts of a program or different programs.Imagine you have a message board where people post messages, and your program needs to read those messages one by one.

🎯 Goal: Create a Java Kafka consumer client that connects to a Kafka server, subscribes to a topic named test-topic, reads messages, and prints them to the console.

📋 What You'll Learn

Create a Properties object with Kafka consumer settings

Create a KafkaConsumer<String, String> object using the properties

Subscribe the consumer to the topic test-topic

Poll messages from Kafka and print each message's key and value

Close the consumer after reading messages

💡 Why This Matters

🌍 Real World

Kafka consumers are used in real applications to process streams of data like logs, user activity, or sensor data in real time.

💼 Career

Knowing how to write Kafka consumer clients in Java is important for backend developers working with distributed systems and real-time data processing.

1

Setup Kafka consumer properties

Create a Properties object called props and set these exact properties:
bootstrap.servers to "localhost:9092",
group.id to "test-group",
key.deserializer to "org.apache.kafka.common.serialization.StringDeserializer",
value.deserializer to "org.apache.kafka.common.serialization.StringDeserializer".

Kafka

import java.util.Properties;

// Your code here

Need a hint?

Use props.put(key, value) to set each property.

2

Create KafkaConsumer and subscribe to topic

Create a KafkaConsumer<String, String> called consumer using props. Then subscribe consumer to the topic "test-topic" using consumer.subscribe().

Kafka

import org.apache.kafka.clients.consumer.KafkaConsumer;
import java.util.Collections;

// Your code here

Need a hint?

Use Collections.singletonList("test-topic") to subscribe to one topic.

3

Poll messages and print key and value

Use consumer.poll() with a timeout of 1000 milliseconds to get records. Then use a for loop with variables record to iterate over records. Inside the loop, print the message key and value using record.key() and record.value().

Kafka

// Your code here

Need a hint?

Use Duration.ofMillis(1000) for the poll timeout.

4

Close the consumer and print output

Call consumer.close() to close the consumer. Then print "Finished reading messages" using System.out.println().

Kafka

// Your code here

Need a hint?

Remember to close the consumer to free resources.