At-most-once delivery in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding At-most-once Delivery in Kafka

Which statement best describes at-most-once delivery semantics in Kafka?

A Messages are delivered only if the consumer commits offsets manually.

B Messages are guaranteed to be delivered at least once, possibly multiple times.

C Messages are delivered exactly once, with no duplicates.

D Messages are delivered zero or one time, with no retries on failure.

💡 Hint

Think about what happens if a message fails to be processed or acknowledged.

At-most-once delivery means messages can be lost but never duplicated. Kafka sends messages without retrying on failure, so a message might be delivered zero or one time.

❓ Predict Output

intermediate

Output of Kafka Producer with At-most-once Settings

Given the following Kafka producer configuration snippet, what is the expected delivery guarantee?

props.put("enable.idempotence", "false");
props.put("acks", "0");
props.put("retries", 0);

What is the expected behavior?

A Messages are retried until acknowledged; at-least-once delivery.

B Messages may be lost but will not be duplicated; at-most-once delivery.

C Messages are delivered exactly once with no duplicates.

D Messages are only sent after manual commit.

💡 Hint

Consider what disabling idempotence and setting acks to 0 means.

With acks=0, the producer does not wait for any acknowledgment, and with retries=0 and idempotence disabled, messages may be lost but will not be duplicated, matching at-most-once semantics.

❓ Predict Output

advanced

Kafka Consumer Offset Behavior with At-most-once Delivery

Consider this Kafka consumer code snippet:

consumer.subscribe(List.of("topic"));
while (true) {
  var records = consumer.poll(Duration.ofMillis(100));
  for (var record : records) {
    System.out.println(record.value());
  }
  consumer.commitSync();
}

If the commitSync() call is moved before processing the records, what delivery guarantee does this provide?

A Exactly-once delivery, because commitSync ensures atomicity.

B At-least-once delivery, because offsets are committed after processing.

C At-most-once delivery, because offsets are committed before processing.

D No delivery guarantee, because commitSync is not used.

💡 Hint

Think about what happens if the consumer crashes after committing but before processing.

Committing offsets before processing means if the consumer crashes after commit but before processing, messages are lost, so at-most-once delivery is achieved.

🔧 Debug

advanced

Identify the Error in At-most-once Kafka Producer Code

What error will this Kafka producer code cause?

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("acks", "0");
props.put("enable.idempotence", "true");
props.put("retries", 0);
KafkaProducer<String, String> producer = new KafkaProducer<>(props);
producer.send(new ProducerRecord<>("topic", "key", "value"));
producer.close();

A IllegalArgumentException because idempotence requires retries > 0.

B No error; code runs with at-most-once delivery.

C TimeoutException because acks=0 disables idempotence.

D NullPointerException due to missing serializers.

💡 Hint

Check Kafka producer configuration rules for idempotence.

Enabling idempotence requires retries to be greater than zero. Setting retries to 0 with idempotence enabled causes an IllegalArgumentException.

🚀 Application

expert

Designing At-most-once Delivery with Kafka Consumer

You want to implement a Kafka consumer that guarantees at-most-once delivery. Which of the following code snippets correctly implements this behavior?

A

consumer.subscribe(List.of("topic"));
while (true) {
  var records = consumer.poll(Duration.ofMillis(100));
  consumer.commitSync();
  for (var record : records) {
    process(record.value());
  }
}

B

consumer.subscribe(List.of("topic"));
while (true) {
  var records = consumer.poll(Duration.ofMillis(100));
  for (var record : records) {
    process(record.value());
  }
  consumer.commitSync();
}

C

consumer.subscribe(List.of("topic"));
while (true) {
  var records = consumer.poll(Duration.ofMillis(100));
  for (var record : records) {
    process(record.value());
    consumer.commitSync();
  }
}

D

consumer.subscribe(List.of("topic"));
while (true) {
  var records = consumer.poll(Duration.ofMillis(100));
  for (var record : records) {
    process(record.value());
  }
  consumer.commitAsync();
}

💡 Hint

Remember that committing offsets before processing leads to at-most-once delivery.

Option A commits offsets before processing, so if the consumer crashes after commit but before processing, messages are lost, ensuring at-most-once delivery.

Practice

1.

What does at-most-once delivery guarantee in Kafka messaging?

easy

A. Messages are sent at least once, possibly duplicated

B. Messages are sent exactly once with no loss

C. Messages are sent only once but some may be lost

D. Messages are sent multiple times intentionally

Solution

1. Step 1: Understand at-most-once delivery meaning

   At-most-once delivery means messages are sent once without retries, so duplicates are avoided but some messages might be lost.

2. Step 2: Compare with other delivery guarantees

   At-least-once allows duplicates, exactly-once avoids loss and duplicates, so only at-most-once fits the description of possible loss but no duplicates.

3. Final Answer:

   Messages are sent only once but some may be lost -> Option C

4. Quick Check:

   At-most-once = no duplicates, possible loss [OK]

Hint: At-most-once means no retries, so possible message loss [OK]

Common Mistakes:

• Confusing at-most-once with exactly-once delivery
• Assuming no messages are ever lost
• Thinking duplicates can occur in at-most-once

2.

Which Kafka producer configuration enables at-most-once delivery?

producer = KafkaProducer(acks=?, retries=? )

easy

A. acks=all, retries=5

B. acks=0, retries=0

C. acks=1, retries=3

D. acks=all, retries=0

Solution

1. Step 1: Identify settings for at-most-once

   At-most-once requires no retries and no acknowledgments, so acks=0 and retries=0 are needed.

2. Step 2: Check other options

   Other options enable retries or acknowledgments, which allow retries and possible duplicates, not at-most-once.

3. Final Answer:

   acks=0, retries=0 -> Option B

4. Quick Check:

   acks=0 & retries=0 = at-most-once [OK]

Hint: Set acks and retries to zero for at-most-once [OK]

Common Mistakes:

• Setting retries > 0 allows retries, not at-most-once
• Using acks=all causes waiting for confirmation
• Confusing acks=1 with acks=0

3.

Given the following Kafka producer code snippet, what is the expected delivery guarantee?

producer = KafkaProducer(acks=0, retries=0)
producer.send('topic', b'message')

medium

A. At-most-once delivery with possible message loss

B. No message will be sent

C. Exactly-once delivery with no duplicates

D. At-least-once delivery with possible duplicates

Solution

1. Step 1: Analyze producer settings

   With acks=0 and retries=0, the producer does not wait for acknowledgment and does not retry on failure.

2. Step 2: Determine delivery guarantee

   This means messages may be lost if sending fails, but duplicates won't occur, matching at-most-once delivery.

3. Final Answer:

   At-most-once delivery with possible message loss -> Option A

4. Quick Check:

   acks=0 & retries=0 = at-most-once [OK]

Hint: acks=0 and retries=0 means no retries, so at-most-once [OK]

Common Mistakes:

• Assuming messages are always delivered
• Confusing at-most-once with exactly-once
• Thinking duplicates can happen here

4.

Identify the error in this Kafka producer configuration for at-most-once delivery:

producer = KafkaProducer(acks=1, retries=0)
producer.send('topic', b'message')

medium

A. acks=1 causes waiting for leader ack, not at-most-once

B. retries=0 causes infinite retries

C. Missing enable.idempotence=True

D. Topic name must be uppercase

Solution

1. Step 1: Check acks setting

   acks=1 means the producer waits for leader acknowledgment, allowing retries and possible duplicates, not at-most-once.

2. Step 2: Verify retries and other settings

   Retries=0 disables retries, but acks=1 still waits for confirmation, so this is not pure at-most-once.

3. Final Answer:

   acks=1 causes waiting for leader ack, not at-most-once -> Option A

4. Quick Check:

   acks must be 0 for at-most-once [OK]

Hint: acks must be 0 for at-most-once delivery [OK]

Common Mistakes:

• Thinking retries=0 means no retries but ignoring acks effect
• Assuming idempotence is required for at-most-once
• Believing topic names must be uppercase

5.

You want to configure a Kafka producer for at-most-once delivery but also need to minimize message loss. Which combination of settings is best?

Options:
A) acks=1, retries=0
B) acks=0, retries=0
C) acks=0, retries=3
D) acks=all, retries=0

hard

A. acks=all, retries=0 (wait for all acks, no retries)

B. acks=0, retries=0 (pure at-most-once, possible loss)

C. acks=0, retries=3 (retries with no ack, risky duplicates)

D. acks=1, retries=0 (wait for ack, no retries, less loss)

Solution

1. Step 1: Understand trade-offs

   Pure at-most-once (acks=0, retries=0) risks message loss. Waiting for ack reduces loss but may not be pure at-most-once.

2. Step 2: Evaluate options

   acks=1, retries=0 (wait for ack, no retries, less loss) waits for leader ack but disables retries, reducing loss while avoiding duplicates, closest to at-most-once with less loss.

3. Final Answer:

   acks=1, retries=0 (wait for ack, no retries, less loss) -> Option D

4. Quick Check:

   acks=1 & retries=0 balances loss and duplicates [OK]

Hint: Wait for ack but disable retries to reduce loss in at-most-once [OK]

Common Mistakes:

• Choosing retries > 0 breaks at-most-once
• Ignoring ack setting impact on loss
• Assuming acks=all with no retries is at-most-once