At-least-once delivery in Kafka - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to commit the offset after processing the message.

Kafka

consumer.commitSync([1]);

Drag options to blanks, or click blank then click option'

Anull

Btrue

Cfalse

Doffsets

Attempts:

3 left

💡 Hint

Common Mistakes

Passing true or false instead of offsets causes errors.

Not committing offsets can lead to message reprocessing.

The commitSync method requires the offsets to commit, so 'offsets' is the correct argument.

2fill in blank

medium

Complete the code to enable auto-commit for at-least-once delivery.

Kafka

"enable.auto.commit": [1],

Drag options to blanks, or click blank then click option'

A"false"

BTrue

CFalse

D"true"

Attempts:

3 left

💡 Hint

Common Mistakes

Using string 'true' instead of boolean true causes config issues.

Setting auto-commit to false disables automatic commits.

Setting 'enable.auto.commit' to true enables automatic offset commits for at-least-once delivery.

3fill in blank

hard

Fix the error in the code to process messages and commit offsets synchronously.

Kafka

while (true) {
    ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
    for (ConsumerRecord<String, String> record : records) {
        process(record);
    }
    consumer.[1]();
}

Drag options to blanks, or click blank then click option'

AcommitOffset

BcommitAsync

CcommitSync

Dcommit

Attempts:

3 left

💡 Hint

Common Mistakes

Using commitAsync() can cause message duplication.

commit() and commitOffset() are not valid Kafka consumer methods.

commitSync() commits offsets synchronously, ensuring at-least-once delivery guarantees.

4fill in blank

hard

Fill both blanks to create a dictionary comprehension that stores offsets for partitions with offset greater than 5.

Kafka

Map<TopicPartition, OffsetAndMetadata> offsets = records.partitions().stream()
    .collect(Collectors.toMap(
        partition -> partition,
        partition -> new OffsetAndMetadata([1])
    ));

if (offsets.entrySet().stream().anyMatch(entry -> entry.getValue().offset() [2] 5)) {
    consumer.commitSync(offsets);
}

Drag options to blanks, or click blank then click option'

Arecords.records(partition).get(records.records(partition).size() - 1).offset() + 1

Brecords.records(partition).get(0).offset()

C>

D<

Attempts:

3 left

💡 Hint

Common Mistakes

Using the first record's offset instead of the last.

Using '<' instead of '>' in the condition.

The offset stored is the last record's offset plus one, and the condition checks if offset is greater than 5.

5fill in blank

hard

Fill all three blanks to create a Python dictionary comprehension that stores offsets for words longer than 3 characters.

Kafka

offsets = {word[1]: [2] for word in words if len(word) [3] 3}

Drag options to blanks, or click blank then click option'

A.upper()

Blen(word)

C>

D<=

Attempts:

3 left

💡 Hint

Common Mistakes

Using '<=' instead of '>' in the condition.

Using the word itself instead of its uppercase form as key.

The comprehension uses the uppercase word as key, length as value, and filters words longer than 3 characters.

Practice

1. What does at-least-once delivery guarantee in Kafka?

easy

A. Messages are delivered only once and never retried

B. Messages are delivered exactly once with no duplicates

C. Messages are never lost but might be delivered more than once

D. Messages might be lost but never duplicated

Solution

1. Step 1: Understand delivery guarantees

   At-least-once means Kafka ensures messages reach the consumer at least once, so no loss occurs.

2. Step 2: Consider duplicates possibility

   Because retries can happen, duplicates may occur, so messages might be delivered multiple times.

3. Final Answer:

   Messages are never lost but might be delivered more than once -> Option C

4. Quick Check:

   At-least-once = no loss, possible duplicates [OK]

Hint: At-least-once means no loss, duplicates possible [OK]

Common Mistakes:

• Confusing at-least-once with exactly-once delivery
• Thinking messages can be lost
• Assuming no duplicates occur

2. Which Kafka consumer configuration is essential for implementing at-least-once delivery?

easy

A. enable.auto.commit = false

B. enable.auto.commit = true

C. auto.offset.reset = earliest

D. max.poll.records = 1

Solution

1. Step 1: Identify offset commit behavior

   At-least-once requires manual control of offset commits to avoid losing messages.

2. Step 2: Choose correct config for manual commit

   Setting enable.auto.commit = false disables automatic commits, allowing manual commits after processing.

3. Final Answer:

   enable.auto.commit = false -> Option A

4. Quick Check:

   Manual commit enabled by disabling auto commit [OK]

Hint: Disable auto commit for manual offset control [OK]

Common Mistakes:

• Leaving auto commit enabled causes possible message loss
• Confusing auto.offset.reset with commit control
• Assuming max.poll.records affects delivery guarantee

3. Given this Kafka consumer code snippet, what is the expected behavior regarding message delivery?

consumer.subscribe(['topic'])
for message in consumer:
    process(message)
    consumer.commit()

medium

A. Messages are processed at least once, duplicates possible

B. Messages may be lost if commit happens before processing

C. Messages are processed exactly once with no duplicates

D. Messages are never committed, causing infinite reprocessing

Solution

1. Step 1: Analyze commit timing

   Commit happens after processing each message, so offsets are saved only after successful processing.

2. Step 2: Understand delivery guarantee

   If a failure occurs before commit, message will be reprocessed, causing duplicates but no loss.

3. Final Answer:

   Messages are processed at least once, duplicates possible -> Option A

4. Quick Check:

   Commit after processing = at-least-once delivery [OK]

Hint: Commit after processing ensures no loss, duplicates possible [OK]

Common Mistakes:

• Assuming commit before processing
• Thinking duplicates cannot happen
• Believing messages are lost on failure

4. Identify the bug in this Kafka consumer code that aims for at-least-once delivery:

consumer.subscribe(['topic'])
for message in consumer:
    consumer.commit()
    process(message)

medium

A. Using subscribe instead of assign causes duplicates

B. Committing offset before processing causes message loss on failure

C. Not committing offsets causes infinite reprocessing

D. process() function is missing a return statement

Solution

1. Step 1: Check commit position relative to processing

   Offsets are committed before processing, so if processing fails, message is lost.

2. Step 2: Understand at-least-once requirement

   To guarantee at-least-once, commit must happen after successful processing to avoid loss.

3. Final Answer:

   Committing offset before processing causes message loss on failure -> Option B

4. Quick Check:

   Commit after processing to avoid loss [OK]

Hint: Commit offsets only after processing completes [OK]

Common Mistakes:

• Committing before processing causes data loss
• Ignoring commit timing importance
• Confusing subscribe with assign usage

5. You want to ensure at-least-once delivery in Kafka but avoid processing duplicates as much as possible. Which approach best achieves this?

hard

A. Use exactly-once semantics without any manual offset control

B. Enable auto commit and process messages without tracking offsets

C. Commit offsets before processing to speed up throughput

D. Manually commit offsets after processing and implement idempotent processing logic

Solution

1. Step 1: Combine manual commit with processing logic

   Manual commit after processing ensures no message loss; idempotent logic handles duplicates safely.

2. Step 2: Evaluate other options

   Auto commit risks loss; committing before processing risks loss; exactly-once semantics require special setup.

3. Final Answer:

   Manually commit offsets after processing and implement idempotent processing logic -> Option D

4. Quick Check:

   Manual commit + idempotency = best at-least-once practice [OK]

Hint: Use manual commit plus idempotent processing to reduce duplicates [OK]

Common Mistakes:

• Relying on auto commit for at-least-once
• Committing offsets before processing
• Ignoring idempotent processing importance