At-most-once delivery in Kafka - Step-by-Step Execution

Process Flow - At-most-once delivery

Message Produced

↓

Send to Kafka Broker

↓

Message Stored in Partition

↓

Consumer Polls Message

↓

Commit Offset

↓

Process Message

↓

Next Message

At-most-once delivery means messages are sent and processed zero or one time, allowing possible message loss but no duplicates.

Execution Sample

Kafka

producer.send(message)
// no retries
consumer.poll()
consumer.commit()
// commit before process
process(message)

This code sends a message without retries, polls and commits offset before processing, risking message loss if crash after commit before process but no duplicates.

Process Table

Step	Action	Message State	Offset Commit	Result
1	Producer sends message	Message sent to broker	No	Message may be lost if failure occurs
2	Consumer polls message	Message received by consumer	No	Consumer has message
3	Commit offset	Offset committed	Yes	Prevents re-delivery on restart
4	Consumer crashes before process	Message not processed	Yes	Message lost, no re-delivery
5	Consumer restarts, next poll	No message re-delivered	Yes	At-most-once delivery achieved

💡 Consumer crash after offset commit (before process) causes message loss, no re-delivery.

Status Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	Final
message_state	Not sent	Sent to broker	Received by consumer	Offset committed	Not processed (crash)	Lost (skipped)
offset_commit	Not committed	Not committed	Not committed	Committed	Committed	Committed

Key Moments - 3 Insights

Why can messages be lost in at-most-once delivery?

Because the consumer may commit the offset after polling but crash before processing, so Kafka skips the message on restart (see execution_table step 4).

Does at-most-once delivery guarantee no duplicate messages?

Yes, because offset committed before processing prevents re-delivery even if processing fails, no producer retries (see execution_table step 5).

What happens if the producer retries sending messages?

Retries can cause duplicates, so at-most-once delivery usually disables retries to avoid duplicates (not shown in this trace).

Visual Quiz - 3 Questions

Look at the execution table, at which step does the consumer crash (after commit before process)?

A Step 2

B Step 3

C Step 4

D Step 5

💡 Hint

Check the 'Consumer crashes before process' description in step 4 of the execution_table.

According to the variable tracker, what is the offset_commit state after step 3?

A Not committed

B Committed

C Partially committed

D Unknown

💡 Hint

Look at the offset_commit row in variable_tracker after step 3.

If the producer enabled retries, how would the at-most-once delivery behavior change?

A Duplicates could occur

B No change in delivery guarantees

C Messages might be lost more often

D Messages would be delivered exactly once

💡 Hint

Refer to key_moments about retries causing duplicates.

Concept Snapshot

At-most-once delivery:
- Messages sent once, no retries
- Consumer commits offset before process
- Possible message loss, no duplicates
- Fast but less reliable
- Use when losing messages is acceptable

Full Transcript

At-most-once delivery in Kafka means messages are sent and processed zero or one time. The producer sends messages without retries. The consumer polls, commits offset before processing. If the consumer crashes after commit before processing, the message is lost and not re-delivered. This avoids duplicates but risks losing messages. The execution table shows each step from sending to the crash scenario. Variable tracking shows message state and offset commit status. Key moments clarify why messages can be lost and why duplicates do not occur. The quiz tests understanding of crash timing, offset commit state, and retry effects. This delivery mode is fast but less reliable, suitable when losing some messages is acceptable.