Retention policies (time-based, size-based) in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the effect of this retention.ms setting?

Given a Kafka topic configured with retention.ms=60000, what happens to messages older than 60 seconds?

Kafka

kafka-topics.sh --alter --topic example-topic --config retention.ms=60000 --bootstrap-server localhost:9092

A Messages older than 60 seconds are moved to a backup topic.

B Messages are retained indefinitely regardless of age.

C Messages older than 60 seconds are compressed but not deleted.

D Messages older than 60 seconds are deleted automatically.

💡 Hint

Think about what retention.ms controls in Kafka.

The retention.ms setting controls how long Kafka retains messages before deleting them. Setting it to 60000 means messages older than 60 seconds are removed.

❓ Predict Output

intermediate

What happens with this retention.bytes setting?

A Kafka topic has retention.bytes=1048576 (1 MB). What happens when the topic size exceeds 1 MB?

Kafka

kafka-topics.sh --alter --topic example-topic --config retention.bytes=1048576 --bootstrap-server localhost:9092

A Oldest messages are deleted to keep the topic size under 1 MB.

B New messages are rejected once the topic size exceeds 1 MB.

C The topic size grows indefinitely ignoring retention.bytes.

D Messages are archived to disk when size exceeds 1 MB.

💡 Hint

Consider what retention.bytes limits in Kafka.

The retention.bytes setting limits the total size of the topic. When the size exceeds this limit, Kafka deletes the oldest messages to free space.

🧠 Conceptual

advanced

Which retention policy combination causes messages to be deleted earliest?

If a Kafka topic has retention.ms=3600000 (1 hour) and retention.bytes=10485760 (10 MB), when are messages deleted?

A Messages are deleted when either 1 hour passes or topic size exceeds 10 MB, whichever comes first.

B Messages are never deleted automatically with both settings set.

C Messages are deleted only when topic size exceeds 10 MB regardless of time.

D Messages are deleted only after 1 hour regardless of size.

💡 Hint

Think about how Kafka applies retention.ms and retention.bytes together.

Kafka deletes messages when either the time limit or size limit is reached, whichever happens first.

❓ Predict Output

advanced

What error occurs with this retention.ms value?

What happens if you set retention.ms=-1000 for a Kafka topic?

Kafka

kafka-topics.sh --alter --topic example-topic --config retention.ms=-1000 --bootstrap-server localhost:9092

A Kafka deletes all messages immediately upon setting retention.ms negative.

B Kafka throws a ConfigException due to invalid negative retention.ms value.

C Kafka treats negative retention.ms as infinite retention (never delete).

D Kafka ignores the retention.ms setting and uses default retention.

💡 Hint

Consider valid ranges for retention.ms in Kafka.

Kafka does not allow negative values for retention.ms and throws a configuration error.

🧠 Conceptual

expert

How does Kafka handle retention with log compaction enabled?

If a Kafka topic has both retention.ms=86400000 (1 day) and log compaction enabled, what is the retention behavior?

A Kafka deletes all messages after 1 day regardless of compaction.

B Kafka ignores retention.ms and only uses log compaction to delete messages.

C Kafka deletes messages older than 1 day but retains the latest record for each key indefinitely.

D Kafka disables log compaction when retention.ms is set.

💡 Hint

Think about how log compaction and retention.ms work together.

With log compaction, Kafka retains the latest record for each key indefinitely, but messages older than retention.ms without a newer key version are deleted.