What Is Tombstone in Kafka: Explanation and Usage

In Kafka, a tombstone is a special message with a key and a null value that marks a record as deleted. It signals Kafka to remove the record during log compaction, helping manage data lifecycle efficiently.

How It Works

Imagine you have a notebook where you write down important notes. If you want to erase a note, you can't just rip out the page; instead, you put a big "X" on it to mark it as deleted. In Kafka, a tombstone message works like that "X". It is a message with the same key as the original record but with a null value.

When Kafka runs log compaction, it looks for these tombstone messages and removes the original record along with the tombstone itself after some time. This process helps keep the topic clean by deleting outdated or unwanted data without deleting the entire topic or partition.

This mechanism is useful because Kafka topics are append-only logs, so you cannot directly delete a message. Instead, tombstones mark messages as deleted, and Kafka cleans them up later.

Example

This example shows how to produce a tombstone message in Kafka using the Java client. The key is "user123" and the value is null, marking the record for deletion.

java

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import java.util.Properties;

public class TombstoneExample {
    public static void main(String[] args) {
        Properties props = new Properties();
        props.put("bootstrap.servers", "localhost:9092");
        props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

        try (KafkaProducer<String, String> producer = new KafkaProducer<>(props)) {
            // Produce a tombstone message with key "user123" and null value
            ProducerRecord<String, String> tombstone = new ProducerRecord<>("users-topic", "user123", null);
            producer.send(tombstone);
            System.out.println("Tombstone message sent for key user123");
        }
    }
}

Output

Tombstone message sent for key user123

When to Use

Use tombstone messages when you want to delete or invalidate a record in a Kafka topic that uses log compaction. This is common in scenarios like:

• Deleting user profiles or settings when a user account is removed.
• Removing outdated cache entries stored in Kafka topics.
• Marking data as deleted in event sourcing or change data capture systems.

They help keep the topic size manageable and ensure consumers do not see stale data.

Key Points

• A tombstone is a message with a key and a null value.
• It signals Kafka to delete the record during log compaction.
• Kafka topics are append-only, so tombstones mark deletions instead of removing messages immediately.
• Useful for managing data lifecycle in compacted topics.

Key Takeaways

A tombstone in Kafka is a message with a key and null value that marks a record as deleted.

Kafka uses tombstones during log compaction to remove old or deleted records.

Tombstones help manage data lifecycle without deleting entire topics or partitions.

They are essential for use cases like user data deletion and cache invalidation in Kafka.

Producing a tombstone message requires sending a record with the key and null value.