Why At-least-once delivery in Kafka? - Purpose & Use Cases

The Big Idea

What if your messages got lost or repeated without you knowing?

The Scenario

Imagine you are sending important messages one by one by hand, like passing notes in class. Sometimes, you might miss giving a note or accidentally give it twice, causing confusion.

The Problem

Manually ensuring every message is delivered exactly once is slow and tricky. You might lose messages or send duplicates, leading to errors and extra work to fix them.

The Solution

At-least-once delivery in Kafka guarantees every message is delivered one or more times, so none are lost. It handles retries automatically, making sure your messages reach their destination even if problems happen.

Before vs After

✗ Before

sendMessage(msg)
// no retry, no confirmation

✓ After

producer.send(msg).get()
// waits for confirmation, retries if needed

What It Enables

This concept ensures reliable message delivery, so your system never misses important data even during failures.

Real Life Example

In online shopping, at-least-once delivery makes sure every order message reaches the warehouse system, preventing lost orders and unhappy customers.

Key Takeaways

Manual message sending can lose or duplicate data.

At-least-once delivery retries until messages are confirmed.

This improves reliability in critical systems like orders or payments.