What is linger.ms in Kafka Producer and How It Works

linger.ms in Kafka Producer is a setting that controls how long the producer waits before sending a batch of messages. It helps group messages together to improve throughput by delaying sending for a short time to allow more messages to accumulate.

How It Works

Imagine you are mailing letters. Instead of sending each letter immediately, you wait a little to collect more letters and send them all at once. This saves time and postage. linger.ms works the same way for Kafka Producer.

When a message is ready to send, the producer waits up to the time set by linger.ms to collect more messages into a batch. If the batch fills up before the time ends, it sends immediately. Otherwise, it sends after the wait time.

This delay helps increase efficiency by reducing the number of requests sent to the Kafka broker, improving throughput but adding a small delay to message delivery.

Example

This example shows how to set linger.ms in a Kafka Producer configuration using Java. It sets the linger time to 100 milliseconds, meaning the producer waits up to 100 ms to batch messages before sending.

java

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
import java.util.Properties;

public class ProducerExample {
    public static void main(String[] args) {
        Properties props = new Properties();
        props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
        props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer");
        props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer");
        props.put(ProducerConfig.LINGER_MS_CONFIG, 100); // wait up to 100 ms before sending

        KafkaProducer<String, String> producer = new KafkaProducer<>(props);

        for (int i = 0; i < 5; i++) {
            ProducerRecord<String, String> record = new ProducerRecord<>("my-topic", "key" + i, "value" + i);
            producer.send(record);
        }

        producer.close();
    }
}

Output

No direct output; messages are sent to Kafka broker with batching delay up to 100 ms.

When to Use

Use linger.ms when you want to improve Kafka Producer throughput by batching messages together. It is helpful when your application sends many small messages rapidly.

For example, if you have a logging system that generates many log entries per second, setting linger.ms to a small value like 50-100 ms can reduce network overhead and increase efficiency.

However, if your application needs very low latency and cannot tolerate delays, keep linger.ms low or zero to send messages immediately.

Key Points

• linger.ms controls the max wait time to batch messages before sending.
• Higher values increase batching and throughput but add delay.
• Lower or zero values reduce delay but may send smaller batches.
• It balances latency and throughput in Kafka Producer.

Key Takeaways

linger.ms sets how long Kafka Producer waits to batch messages before sending.

Increasing linger.ms improves throughput by sending larger batches.

Lower linger.ms reduces delay but may increase network calls.

Use it to balance message latency and producer efficiency based on your needs.