What Is Header in Kafka Message: Explanation and Example

In Kafka, a header is a key-value pair attached to a message that carries metadata separate from the message body. Headers help add extra information like message type or tracing data without changing the main message content.

How It Works

Think of a Kafka message like a letter you send in the mail. The main message body is the letter itself, while the header is like the envelope's label that tells the post office extra details such as the sender's address or special handling instructions.

In Kafka, headers are small pieces of data attached to each message as key-value pairs. They travel along with the message but are separate from the main content. This allows producers and consumers to add or read metadata without changing the actual message payload.

This mechanism helps systems pass extra context, like message version, content type, or tracing IDs, which can be useful for routing, debugging, or processing decisions.

Example

This example shows how to add and read headers in a Kafka message using Java Kafka client.

java

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.clients.producer.RecordMetadata;
import org.apache.kafka.common.header.internals.RecordHeader;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import java.util.Properties;
import java.util.Collections;

public class KafkaHeaderExample {
    public static void main(String[] args) throws Exception {
        // Producer setup
        Properties producerProps = new Properties();
        producerProps.put("bootstrap.servers", "localhost:9092");
        producerProps.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        producerProps.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        KafkaProducer<String, String> producer = new KafkaProducer<>(producerProps);

        // Create a record with a header
        ProducerRecord<String, String> record = new ProducerRecord<>("my-topic", "key1", "Hello Kafka");
        record.headers().add(new RecordHeader("source", "sensor1".getBytes()));

        // Send the record
        producer.send(record, (RecordMetadata metadata, Exception e) -> {
            if (e == null) {
                System.out.println("Message sent with header to partition " + metadata.partition());
            } else {
                e.printStackTrace();
            }
        });
        producer.close();

        // Consumer setup
        Properties consumerProps = new Properties();
        consumerProps.put("bootstrap.servers", "localhost:9092");
        consumerProps.put("group.id", "group1");
        consumerProps.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        consumerProps.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        KafkaConsumer<String, String> consumer = new KafkaConsumer<>(consumerProps);
        consumer.subscribe(Collections.singletonList("my-topic"));

        // Poll and read headers
        ConsumerRecords<String, String> records = consumer.poll(java.time.Duration.ofMillis(1000));
        for (ConsumerRecord<String, String> rec : records) {
            System.out.println("Received message: " + rec.value());
            rec.headers().forEach(header -> {
                System.out.println("Header key: " + header.key() + ", value: " + new String(header.value()));
            });
        }
        consumer.close();
    }
}

Output

Message sent with header to partition 0
Received message: Hello Kafka
Header key: source, value: sensor1

When to Use

Use headers in Kafka messages when you want to send extra information that helps consumers understand or process the message better without changing the main data. For example:

• Adding message type or version to handle different formats.
• Including tracing IDs for monitoring and debugging distributed systems.
• Passing routing or priority information to downstream services.
• Attaching security tokens or metadata for authorization checks.

Headers keep your message payload clean and focused on the main data, while still allowing flexible metadata exchange.

Key Points

• Headers are key-value pairs attached to Kafka messages as metadata.
• They travel with the message but are separate from the main payload.
• Headers help add context like message type, tracing info, or routing data.
• They are useful for flexible, non-intrusive metadata exchange.
• Kafka clients support adding and reading headers easily.

Key Takeaways

Kafka message headers carry metadata as key-value pairs separate from the main message body.

Headers help add context like tracing IDs or message versions without changing the payload.

Use headers to pass extra info for routing, debugging, or processing decisions.

Kafka clients provide simple APIs to add and read headers on messages.

Headers keep your message data clean while enabling flexible metadata exchange.