Log compaction in Kafka - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is log compaction in Kafka?

Log compaction is a Kafka feature that retains only the latest value for each key in a topic, removing older duplicates to save storage and ensure data freshness.

beginner

How does Kafka decide which messages to keep during log compaction?

Kafka keeps the latest message for each unique key and removes older messages with the same key during log compaction.

intermediate

What is the difference between log compaction and log retention by time?

Log retention by time deletes messages older than a set time, while log compaction keeps the latest message per key regardless of age.

intermediate

Why is log compaction useful in Kafka?

It helps keep topics small by removing outdated data, supports stateful stream processing, and ensures consumers can recover the latest state efficiently.

advanced

What happens if a message with a key is sent with a null value in a compacted topic?

A message with a null value acts as a delete marker (tombstone) for that key, so the key and its data are removed during compaction.

What does Kafka's log compaction keep for each key?

A The latest message only

B All messages regardless of key

C Only messages older than a set time

D Messages with null values only

Log compaction keeps only the latest message for each unique key.

What is the purpose of a tombstone message in a compacted topic?

A To add a new key

B To mark a key for deletion

C To duplicate a message

D To reset the topic

A tombstone message has a null value and marks the key for deletion during compaction.

Which scenario best fits using log compaction?

A Archiving logs to cold storage

B Storing all sensor readings forever

C Deleting messages after 7 days

D Keeping the latest user profile updates

Log compaction is ideal for keeping the latest state like user profiles.

How does log compaction affect topic size?

A It increases size by adding more messages

B It has no effect on size

C It reduces size by removing old duplicates

D It deletes all messages periodically

Log compaction removes older duplicates, reducing topic size.

Can log compaction remove messages with unique keys that have never been duplicated?

A Yes, if the message value is null

B Yes, it removes all messages older than a threshold

C No, it keeps the latest message for each key

D No, it removes only messages with duplicate keys

Messages with null values act as tombstones and are removed during compaction.

Explain how Kafka's log compaction works and why it is useful.

Think about how Kafka keeps only the freshest data for each key.

Describe the role of tombstone messages in log compaction.

Consider how Kafka deletes keys using special messages.