Producer API basics in Kafka - Mini Project: Build & Apply

Producer API basics

📖 Scenario: You are working on a simple messaging system where you need to send messages to a Kafka topic. This is common in real-world applications like logging systems, event tracking, or communication between services.

🎯 Goal: Build a Kafka producer that sends a few messages to a specific Kafka topic using the Producer API basics.

📋 What You'll Learn

Create a Kafka producer with the correct configuration

Send messages to a Kafka topic named test-topic

Use a loop to send multiple messages

Print confirmation after sending each message

💡 Why This Matters

🌍 Real World

Kafka producers are used in many real-world systems to send data streams like logs, user activity, or sensor data to other services for processing.

💼 Career

Understanding Kafka producer basics is essential for roles in backend development, data engineering, and real-time data processing.

1

Create Kafka producer configuration

Create a Properties object called props and set the following properties exactly: bootstrap.servers to localhost:9092, key.serializer to org.apache.kafka.common.serialization.StringSerializer, and value.serializer to org.apache.kafka.common.serialization.StringSerializer.

Kafka

# Your code here

Hint

Use props.put(key, value) to set each configuration property.

2

Create Kafka producer instance

Create a KafkaProducer<String, String> object called producer using the props configuration.

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
// Your code here

Hint

Use new KafkaProducer<>(props) to create the producer.

3

Send messages to the topic

Use a for loop with an integer variable i from 1 to 3 inclusive. Inside the loop, send a message to the topic test-topic with the value "Message " + i using producer.send() and a ProducerRecord<String, String>. Use i as the key.

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
KafkaProducer producer = new KafkaProducer(props);
// Your code here

Hint

Use new ProducerRecord<>(topic, key, value) inside producer.send().

4

Print confirmation after sending

Inside the for loop, after sending each message, print "Sent message: Message " + i. After the loop, close the producer using producer.close().

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
KafkaProducer producer = new KafkaProducer(props);
for (int i = 1; i ("test-topic", Integer.toString(i), "Message " + i));
    // Your code here
}
// Your code here

Hint

Use System.out.println() inside the loop and producer.close() after the loop.

Practice

1. What is the main purpose of the Kafka Producer API?

easy

A. To send messages to Kafka topics

B. To read messages from Kafka topics

C. To manage Kafka cluster configurations

D. To monitor Kafka broker health

Solution

1. Step 1: Understand Kafka Producer role

   The Producer API is designed to send data to Kafka topics, acting as the message sender.

2. Step 2: Differentiate from other Kafka APIs

   Reading messages is done by the Consumer API, not the Producer API.

3. Final Answer:

   To send messages to Kafka topics -> Option A

4. Quick Check:

   Producer API = send messages [OK]

Hint: Producer API sends data; Consumer API reads data [OK]

Common Mistakes:

• Confusing Producer with Consumer API
• Thinking Producer manages cluster
• Assuming Producer monitors brokers

2. Which of the following is the correct way to create a Kafka Producer in Java?

easy

A. Producer<String> producer = new Producer(props);

B. KafkaProducer producer = new KafkaProducer<>(props);

C. KafkaProducer producer = new KafkaProducer();

D. KafkaProducer producer = KafkaProducer.create(props);

Solution

1. Step 1: Recall KafkaProducer constructor usage

   The KafkaProducer class requires properties and type parameters for key and value types.

2. Step 2: Check each option's syntax

   KafkaProducer producer = new KafkaProducer<>(props); correctly uses generics and constructor with properties. Others miss generics, constructor, or use invalid methods.

3. Final Answer:

   KafkaProducer producer = new KafkaProducer<>(props); -> Option B

4. Quick Check:

   Correct constructor with generics = KafkaProducer producer = new KafkaProducer<>(props); [OK]

Hint: Use new KafkaProducer<>(props) with key and value types [OK]

Common Mistakes:

• Omitting generics for key and value
• Using wrong constructor or method
• Not passing properties object

3. What will be the output of the following Java code snippet?

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
KafkaProducer producer = new KafkaProducer<>(props);
ProducerRecord record = new ProducerRecord<>("my-topic", "key1", "value1");
producer.send(record);
System.out.println("Message sent");
producer.close();

medium

A. No output

B. Compilation error

C. Message sent

D. Runtime exception due to missing topic

Solution

1. Step 1: Analyze producer setup and send call

   The code sets up properties correctly, creates a producer, and sends a message to "my-topic" with key and value.

2. Step 2: Check output statement and close

   After sending, it prints "Message sent" and closes the producer. No errors expected if Kafka broker is running.

3. Final Answer:

   Message sent -> Option C

4. Quick Check:

   Successful send prints message [OK]

Hint: If send() succeeds, print statement runs [OK]

Common Mistakes:

• Assuming send() is synchronous and blocks
• Thinking missing topic causes error here
• Ignoring need for running Kafka broker

4. Identify the error in this Kafka Producer code snippet:

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "StringSerializer");
props.put("value.serializer", "StringSerializer");
KafkaProducer producer = new KafkaProducer<>(props);
ProducerRecord record = new ProducerRecord<>("topic", "key", "value");
producer.send(record);
producer.close();

medium

A. Missing topic name in ProducerRecord

B. Properties object not initialized

C. Producer not closed properly

D. Incorrect serializer class names

Solution

1. Step 1: Check serializer class names in properties

   The serializer values must be full class names like "org.apache.kafka.common.serialization.StringSerializer".

2. Step 2: Verify other code parts

   Topic name is present, producer is closed properly, and properties object is initialized correctly.

3. Final Answer:

   Incorrect serializer class names -> Option D

4. Quick Check:

   Serializer class names must be full package names [OK]

Hint: Use full serializer class names with package [OK]

Common Mistakes:

• Using short class names without package
• Forgetting to close producer
• Leaving topic name empty

5. You want to send messages with keys to ensure ordering per key in Kafka. Which ProducerRecord constructor should you use?

hard

A. ProducerRecord(String topic, K key, V value)

B. ProducerRecord(String topic, V value)

C. ProducerRecord(K key, V value)

D. ProducerRecord(String topic)

Solution

1. Step 1: Understand ordering by key in Kafka

   Kafka guarantees ordering within a partition, and keys determine partition assignment.

2. Step 2: Identify correct ProducerRecord constructor

   To send with a key, use the constructor that includes topic, key, and value.

3. Final Answer:

   ProducerRecord(String topic, K key, V value) -> Option A

4. Quick Check:

   Use constructor with topic, key, and value for ordering [OK]

Hint: Include key in ProducerRecord to maintain order [OK]

Common Mistakes:

• Using constructor without key loses ordering
• Omitting topic causes errors
• Confusing key and value parameters