Why Batching and linger configuration in Kafka? - Purpose & Use Cases

The Big Idea

What if you could send many messages at once and make your app lightning fast?

The Scenario

Imagine sending messages one by one over a network, like mailing individual letters instead of a single package.

The Problem

Sending each message separately is slow and wastes resources. It causes delays and overloads the network with many tiny requests.

The Solution

Batching groups many messages together before sending, and linger configuration waits briefly to collect more messages, making sending efficient and faster.

Before vs After

✗ Before

producer.send(message1)
producer.send(message2)
producer.send(message3)

✓ After

producer.send(batch_of_messages)
producer.flush()

What It Enables

This lets your system send many messages quickly and smoothly, saving time and network power.

Real Life Example

A chat app collects many user messages for a short moment, then sends them all at once to reduce delays and save bandwidth.

Key Takeaways

Sending messages one by one is slow and costly.

Batching groups messages to send together efficiently.

Linger waits briefly to gather more messages before sending.