What is acks in Kafka Producer: Explanation and Usage

acks in Kafka producer is a setting that controls how many acknowledgments the producer waits for from Kafka brokers before considering a message sent successfully. It helps balance between data safety and performance by specifying if the producer waits for no acknowledgment, leader acknowledgment, or all replicas to confirm the message.

How It Works

Imagine you are sending a letter and want to know if it was received. The acks setting in Kafka producer works like asking for a receipt confirmation. If you don't wait for any receipt, you just drop the letter and move on quickly. If you wait for the main post office to confirm, you get some assurance. If you wait for all post offices to confirm, you get the highest safety but it takes longer.

In Kafka, the producer sends messages to a broker called the leader. The acks setting tells the producer how many brokers must confirm they got the message before the producer considers it successful. This controls the balance between speed and data safety.

Example

This example shows how to set acks in a Kafka producer using Java. It sets acks to all to wait for all replicas to confirm the message.

java

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.clients.producer.ProducerConfig;
import java.util.Properties;

public class KafkaAcksExample {
    public static void main(String[] args) {
        Properties props = new Properties();
        props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
        props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer");
        props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer");
        props.put(ProducerConfig.ACKS_CONFIG, "all");

        KafkaProducer<String, String> producer = new KafkaProducer<>(props);

        ProducerRecord<String, String> record = new ProducerRecord<>("my-topic", "key1", "Hello Kafka");

        producer.send(record, (metadata, exception) -> {
            if (exception == null) {
                System.out.println("Message sent successfully to partition " + metadata.partition() + " with offset " + metadata.offset());
            } else {
                System.err.println("Error sending message: " + exception.getMessage());
            }
        });

        producer.flush();
        producer.close();
    }
}

Output

Message sent successfully to partition 0 with offset 15

When to Use

Use acks=0 when you want the fastest sending speed and can tolerate some message loss, such as in logging or metrics where occasional loss is acceptable.

Use acks=1 to wait for the leader broker to confirm, balancing speed and reliability for most applications.

Use acks=all when you need the highest data durability and want to ensure all replicas have the message before proceeding, such as in financial transactions or critical data pipelines.

Key Points

• acks=0: No confirmation, fastest but least safe.
• acks=1: Wait for leader confirmation, good balance.
• acks=all: Wait for all replicas, safest but slower.
• Choosing acks affects message durability and latency.
• Proper setting depends on your application's tolerance for data loss and speed needs.

Key Takeaways

acks controls how many Kafka brokers must confirm a message before the producer considers it sent.

Setting acks=all ensures highest data safety by waiting for all replicas.

acks=0 offers fastest sending but risks message loss.

Choose acks based on your application's need for speed versus reliability.

Proper acks setting helps balance performance and data durability in Kafka.