Message key and value in Kafka - Step-by-Step Execution

Process Flow - Message key and value

Create Message

↓

Assign Key and Value

↓

Send to Kafka Topic

↓

Kafka stores message

↓

Consumers read Key and Value

This flow shows how a Kafka message is created with a key and value, sent to a topic, stored, and then read by consumers.

Execution Sample

Kafka

producer.send(new ProducerRecord<>("my-topic", "user1", "Hello Kafka"));

Send a message with key 'user1' and value 'Hello Kafka' to the topic 'my-topic'.

Process Table

Step	Action	Key	Value	Topic	Result
1	Create ProducerRecord	user1	Hello Kafka	my-topic	Record ready to send
2	Send message	user1	Hello Kafka	my-topic	Message sent to Kafka broker
3	Kafka stores message	user1	Hello Kafka	my-topic	Message stored with key and value
4	Consumer reads message	user1	Hello Kafka	my-topic	Consumer receives key and value

💡 Message key and value successfully sent, stored, and consumed.

Status Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4
key	null	user1	user1	user1	user1
value	null	Hello Kafka	Hello Kafka	Hello Kafka	Hello Kafka
topic	null	my-topic	my-topic	my-topic	my-topic

Key Moments - 2 Insights

Why do we need a key in a Kafka message?

The key helps Kafka decide which partition the message goes to and allows consumers to process messages with the same key in order, as shown in step 3 of the execution_table.

Can a Kafka message have a null key?

Yes, a message can have a null key, but then Kafka distributes messages round-robin across partitions. This is different from the example where the key is 'user1' (step 1).

Visual Quiz - 3 Questions

Look at the execution_table, what is the key value at Step 2?

A null

B "Hello Kafka"

C "user1"

D "my-topic"

💡 Hint

Check the 'Key' column in row for Step 2 in execution_table.

At which step does Kafka store the message?

A Step 1

B Step 3

C Step 2

D Step 4

💡 Hint

Look at the 'Action' column in execution_table for the storing action.

If the key was null, how would the partitioning change?

A Messages are distributed round-robin across partitions

B Messages are sent to a single partition only

C Messages go to partitions based on key hash

D Messages are rejected by Kafka

💡 Hint

Refer to key_moments explanation about null keys and partitioning.

Concept Snapshot

Kafka messages have a key and a value.
The key determines the partition and ordering.
The value is the actual data sent.
Messages are sent to a topic.
Consumers read both key and value.
Null keys cause round-robin partitioning.

Full Transcript

This visual execution shows how a Kafka message is created with a key and value, sent to a topic, stored by Kafka, and then read by consumers. The key helps Kafka decide the partition and maintain order for messages with the same key. The value is the main content of the message. The execution table traces each step from creating the message record, sending it, storing it, and consuming it. Variables key, value, and topic are tracked through each step. Key moments clarify why keys are important and what happens if the key is null. The quiz tests understanding of key values at each step, when Kafka stores the message, and the effect of null keys on partitioning.