Log compaction in Kafka - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to set the topic cleanup policy to log compaction.

Kafka

props.put("cleanup.policy", "[1]");

Drag options to blanks, or click blank then click option'

Acompact

Bdelete

Cnone

Darchive

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'delete' instead of 'compact' disables log compaction.

Misspelling the cleanup policy value.

Setting cleanup.policy to compact enables log compaction for the topic.

2fill in blank

medium

Complete the code to create a compacted topic named 'user-updates'.

Kafka

bin/kafka-topics.sh --create --topic user-updates --partitions 3 --replication-factor 1 --config cleanup.policy=[1] --bootstrap-server localhost:9092

Drag options to blanks, or click blank then click option'

Acompact

Bdelete

Carchive

Dnone

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'delete' which enables time-based deletion instead of compaction.

Omitting the cleanup.policy config.

The cleanup.policy=compact config enables log compaction on the topic.

3fill in blank

hard

Fix the error in the code to configure log compaction for an existing topic.

Kafka

bin/kafka-configs.sh --alter --entity-type topics --entity-name user-logs --add-config cleanup.policy=[1] --bootstrap-server localhost:9092

Drag options to blanks, or click blank then click option'

Adelete

Barchive

Ccompact

Dnone

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'delete' which disables compaction.

Using an invalid cleanup policy value.

To enable log compaction, the cleanup.policy must be set to 'compact'.

4fill in blank

hard

Fill both blanks to create a compacted topic with 4 partitions and replication factor 2.

Kafka

bin/kafka-topics.sh --create --topic logs --partitions [1] --replication-factor [2] --config cleanup.policy=compact --bootstrap-server localhost:9092

Drag options to blanks, or click blank then click option'

A4

B2

C3

D1

Attempts:

3 left

💡 Hint

Common Mistakes

Using replication factor 1 which reduces fault tolerance.

Using fewer partitions than required.

The topic is created with 4 partitions and replication factor 2 to ensure scalability and fault tolerance.

5fill in blank

hard

Fill all three blanks to configure a producer to enable idempotence and set retries to 5.

Kafka

props.put("enable.idempotence", [1]);
props.put("acks", "[2]");
props.put("retries", [3]);

Drag options to blanks, or click blank then click option'

Afalse

Ball

C5

Dtrue

Attempts:

3 left

💡 Hint

Common Mistakes

Setting enable.idempotence to false disables safe retries.

Using acks other than 'all' reduces delivery guarantees.

Setting retries to 0 disables retry attempts.

Enabling idempotence (true) with acks=all and retries set to 5 ensures safe and reliable message delivery, which complements log compaction.

Practice

1. What is the main purpose of log compaction in Kafka?

easy

A. To compress messages to save disk space

B. To delete all messages older than a certain time

C. To replicate messages across multiple brokers

D. To keep only the latest message for each key in a topic

Solution

1. Step 1: Understand log compaction concept

   Log compaction keeps the latest message per key, removing older duplicates.

2. Step 2: Compare with other options

   Deleting old messages is retention, compression is different, replication is for fault tolerance.

3. Final Answer:

   To keep only the latest message for each key in a topic -> Option D

4. Quick Check:

   Log compaction = latest message per key [OK]

Hint: Log compaction keeps latest per key, not all old messages [OK]

Common Mistakes:

• Confusing log compaction with time-based retention
• Thinking compression means log compaction
• Mixing replication with compaction

2. Which configuration enables log compaction on a Kafka topic?

easy

A. cleanup.policy=delete

B. cleanup.policy=compact

C. log.compaction.enabled=true

D. retention.ms=0

Solution

1. Step 1: Identify correct cleanup policy

   Log compaction is enabled by setting cleanup.policy=compact on the topic.

2. Step 2: Eliminate incorrect options

   cleanup.policy=delete is default retention, log.compaction.enabled is invalid, retention.ms controls time retention.

3. Final Answer:

   cleanup.policy=compact -> Option B

4. Quick Check:

   Enable compaction with cleanup.policy=compact [OK]

Hint: Remember: compact, not delete, enables log compaction [OK]

Common Mistakes:

• Using cleanup.policy=delete instead of compact
• Assuming a separate 'log.compaction.enabled' setting exists
• Confusing retention.ms with compaction setting

3. Given a compacted Kafka topic with keys and values:
key1: v1, key2: v2, key1: v3, key3: v4
What messages remain after log compaction?

medium

A. key1: v3, key2: v2, key3: v4

B. key1: v1, key2: v2, key3: v4

C. key1: v3, key2: v2

D. key2: v2, key3: v4

Solution

1. Step 1: Identify latest message per key

   For key1, latest is v3; key2 is v2; key3 is v4.

2. Step 2: List messages after compaction

   Only latest per key remain: key1:v3, key2:v2, key3:v4.

3. Final Answer:

   key1: v3, key2: v2, key3: v4 -> Option A

4. Quick Check:

   Latest message per key kept [OK]

Hint: Keep only last message per key after compaction [OK]

Common Mistakes:

• Keeping all messages instead of only latest per key
• Confusing order and keeping first message per key
• Dropping keys incorrectly

4. You set cleanup.policy=compact on a topic but notice old messages are not removed. What is a likely cause?

medium

A. The topic has no keys in messages

B. The retention.ms is set to 0

C. The broker is running in debug mode

D. The topic has too many partitions

Solution

1. Step 1: Understand log compaction requirement

   Log compaction requires messages to have keys to identify duplicates.

2. Step 2: Analyze why old messages remain

   If messages lack keys, compaction cannot remove duplicates, so old messages stay.

3. Final Answer:

   The topic has no keys in messages -> Option A

4. Quick Check:

   Compaction needs keys to remove old messages [OK]

Hint: Compaction needs keys; no keys means no compaction [OK]

Common Mistakes:

• Assuming retention.ms=0 disables compaction
• Blaming debug mode for compaction issues
• Thinking partition count affects compaction directly

5. You want to store the current state of user profiles in Kafka using log compaction. Which approach is best?

hard

A. Use a topic with cleanup.policy=delete and produce messages without keys

B. Use a topic with retention.ms=0 and produce messages with random keys

C. Use a topic with cleanup.policy=compact and produce messages with user ID as key

D. Use a topic with cleanup.policy=compact but produce messages without keys

Solution

1. Step 1: Identify requirements for storing current state

   Current state needs latest update per user, so keys must be user IDs.

2. Step 2: Choose correct topic configuration

   Log compaction enabled with cleanup.policy=compact keeps latest per key.

3. Step 3: Eliminate wrong options

   Delete policy removes old messages by time, no keys prevent compaction, random keys break state tracking.

4. Final Answer:

   Use a topic with cleanup.policy=compact and produce messages with user ID as key -> Option C

5. Quick Check:

   Compaction + keys = current state storage [OK]

Hint: Use compact policy and keys for current state topics [OK]

Common Mistakes:

• Using delete policy for state storage
• Producing messages without keys
• Using random keys that break compaction logic