Java producer client in Kafka - Step-by-Step Execution

Process Flow - Java producer client

Create Properties

↓

Create KafkaProducer

↓

Create ProducerRecord

↓

Send Record asynchronously

↓

Callback on completion

↓

Close Producer

This flow shows how a Java Kafka producer client is created, sends a message asynchronously, handles the callback, and then closes.

Execution Sample

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
KafkaProducer<String, String> producer = new KafkaProducer<>(props);
ProducerRecord<String, String> record = new ProducerRecord<>("my-topic", "key1", "value1");
producer.send(record, (metadata, exception) -> {
  if (exception == null) System.out.println("Sent");
});
producer.close();

This code creates a Kafka producer, sends a message with key and value to a topic, prints confirmation on success, and closes the producer.

Process Table

Step	Action	State Change	Output
1	Create Properties object	Properties with bootstrap.servers=localhost:9092 and serializers
2	Create KafkaProducer with properties	KafkaProducer ready to send messages
3	Create ProducerRecord with topic=my-topic, key=key1, value=value1	ProducerRecord ready
4	Call producer.send(record, callback)	Send request queued asynchronously
5	Callback triggered after send	No state change	Prints: Sent
6	Call producer.close()	Producer closed, resources freed

💡 Producer closed, no more messages can be sent

Status Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	After Step 6
props	empty	contains bootstrap.servers and serializers	unchanged	unchanged	unchanged	unchanged
producer	null	null	KafkaProducer instance	KafkaProducer instance	KafkaProducer instance	closed
record	null	null	null	ProducerRecord instance	ProducerRecord instance	ProducerRecord instance

Key Moments - 3 Insights

Why does the send method not block the program until the message is sent?

Because send is asynchronous and queues the message to be sent in the background, as shown in step 4 of the execution_table.

What happens if we try to send a message after calling producer.close()?

The producer is closed and cannot send messages anymore, so sending after close will cause an error or no action, as noted in the exit_note.

Why do we provide a callback to the send method?

The callback runs when the send completes, letting us know if it succeeded or failed, as shown in step 5 where it prints 'Sent' on success.

Visual Quiz - 3 Questions

Look at the execution_table, what is the state of the producer variable after step 2?

A closed

B null

C KafkaProducer instance ready to send messages

D ProducerRecord instance

💡 Hint

Check the variable_tracker row for 'producer' after Step 2

At which step does the callback print 'Sent'?

A Step 3

B Step 5

C Step 4

D Step 6

💡 Hint

Look at the output column in the execution_table for the callback action

If we remove the callback from send, what changes in the execution_table?

A Step 5 output would be empty, no confirmation printed

B Producer would not send messages

C Producer would close immediately

D Properties would be empty

💡 Hint

Step 5 shows the callback printing 'Sent'; without callback, no output occurs there

Concept Snapshot

Java Kafka Producer Client:
- Create Properties with bootstrap.servers and serializers
- Instantiate KafkaProducer with Properties
- Create ProducerRecord with topic, key, value
- Send record asynchronously with send(record, callback)
- Callback confirms success or failure
- Close producer to free resources

Full Transcript

This visual execution shows how a Java Kafka producer client works step-by-step. First, we create a Properties object and set the bootstrap server address and serializers. Then, we create a KafkaProducer instance using these properties. Next, we prepare a ProducerRecord with the topic name, key, and value. We send this record asynchronously using the send method, providing a callback that runs when the send completes. The callback prints 'Sent' if the message was sent successfully. Finally, we close the producer to release resources. Variables like props, producer, and record change state as the program runs. The send method does not block; it queues the message to send in the background. The callback lets us know when sending finishes. After closing the producer, no more messages can be sent.

Practice

1. What is the primary role of a Java producer client in Kafka?

easy

A. To send messages to Kafka topics

B. To consume messages from Kafka topics

C. To manage Kafka brokers

D. To monitor Kafka cluster health

Solution

1. Step 1: Understand Kafka client roles

   Kafka has producers that send messages and consumers that receive messages.

2. Step 2: Identify the producer's function

   A Java producer client specifically sends messages to Kafka topics.

3. Final Answer:

   To send messages to Kafka topics -> Option A

4. Quick Check:

   Producer = send messages [OK]

Hint: Producer sends, consumer receives messages [OK]

Common Mistakes:

• Confusing producer with consumer
• Thinking producer manages brokers
• Assuming producer monitors cluster

2. Which of the following is the correct way to set the serializer for the key in a Java Kafka producer?

easy

A. props.add("key.serializer", "StringSerializer");

B. props.set("key.serializer", "StringSerializer");

C. props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");

D. props.put("key.serializer", String.class);

Solution

1. Step 1: Recall how to set properties in Kafka producer

   Kafka producer uses Properties object with put(key, value) method.

2. Step 2: Identify correct serializer setting syntax

   The key.serializer must be set to the full class name as a string.

3. Final Answer:

   props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer"); -> Option C

4. Quick Check:

   Use props.put with full serializer class name [OK]

Hint: Use props.put with full serializer class name string [OK]

Common Mistakes:

• Using props.set or props.add instead of put
• Not using full class name string
• Passing class type instead of string

3. What will be the output of the following Java Kafka producer code snippet?

import java.util.Properties;
import org.apache.kafka.clients.producer.*;
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

Producer<String, String> producer = new KafkaProducer<>(props);
ProducerRecord<String, String> record = new ProducerRecord<>("my-topic", "key1", "value1");
producer.send(record);
producer.close();
System.out.println("Message sent");

medium

A. Message sent

B. Compilation error due to missing imports

C. Runtime error due to missing Kafka server

D. No output

Solution

1. Step 1: Analyze the code flow

   The code creates a producer, sends a message, closes the producer, then prints "Message sent".

2. Step 2: Consider output behavior

   Assuming Kafka server is running and imports are correct, the print statement executes after send and close.

3. Final Answer:

   Message sent -> Option A

4. Quick Check:

   Print after send and close = "Message sent" [OK]

Hint: Print statement runs after send, shows "Message sent" [OK]

Common Mistakes:

• Assuming send() blocks until message delivered
• Ignoring that print runs regardless of Kafka server
• Confusing compile errors with runtime issues

4. Identify the error in this Java Kafka producer code snippet:

import java.util.Properties;
import org.apache.kafka.clients.producer.*;
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

KafkaProducer<String, String> producer = new KafkaProducer<>(props);
ProducerRecord<String, String> record = new ProducerRecord<>("my-topic");
producer.send(record);
producer.close();

medium

A. Producer not closed properly

B. Incorrect serializer class name

C. Missing bootstrap.servers property

D. ProducerRecord missing key argument

Solution

1. Step 1: Check ProducerRecord constructor usage

   The constructor used has only topic, but the generic type expects key and value.

2. Step 2: Identify mismatch in arguments

   ProducerRecord<String, String> expects key and value, but no key/value is provided, causing a compile error.

3. Final Answer:

   ProducerRecord missing key argument -> Option D

4. Quick Check:

   ProducerRecord requires key and value [OK]

Hint: ProducerRecord needs key and value if generics specify both [OK]

Common Mistakes:

• Using wrong ProducerRecord constructor
• Ignoring generic type mismatch
• Assuming value-only constructor works with key type

5. You want to send multiple messages to different Kafka topics using a single Java producer instance efficiently. Which approach is best?

hard

A. Create a new KafkaProducer for each message

B. Reuse one KafkaProducer and send ProducerRecords with different topics

C. Use multiple threads each with its own KafkaProducer

D. Send all messages to one topic and then move them to others

Solution

1. Step 1: Understand KafkaProducer resource usage

   Creating a producer is expensive; reusing one instance is efficient.

2. Step 2: Sending to multiple topics

   ProducerRecords can specify different topics; one producer can send to many topics.

3. Step 3: Evaluate other options

   Creating many producers wastes resources; sending all to one topic is incorrect.

4. Final Answer:

   Reuse one KafkaProducer and send ProducerRecords with different topics -> Option B

5. Quick Check:

   One producer, multiple topics via records [OK]

Hint: Reuse one producer; specify topic per ProducerRecord [OK]

Common Mistakes:

• Creating many producers wastes resources
• Sending all messages to one topic incorrectly
• Assuming each thread needs its own producer