KafkaDebug / FixBeginner · 4 min read

How to Handle Errors in Spring Kafka: Simple Guide

In Spring Kafka, handle errors by configuring an ErrorHandler like SeekToCurrentErrorHandler or DefaultErrorHandler to retry or skip bad messages. You can also use DeadLetterPublishingRecoverer to send failed messages to a dead-letter topic for later inspection.

🔍

Why This Happens

Errors in Spring Kafka consumers happen when message processing fails due to bad data, exceptions in your code, or connectivity issues. Without error handling, the consumer may stop or keep retrying endlessly, blocking message flow.

java

import org.springframework.kafka.annotation.KafkaListener;

public class KafkaConsumer {
    @KafkaListener(topics = "my-topic")
    public void listen(String message) {
        // This will throw an exception for invalid input
        if (message.equals("bad")) {
            throw new RuntimeException("Invalid message");
        }
        System.out.println("Processed: " + message);
    }
}

Output

Exception in thread "kafka-consumer" java.lang.RuntimeException: Invalid message at KafkaConsumer.listen(KafkaConsumer.java:6) ...

🔧

The Fix

Use Spring Kafka's DefaultErrorHandler to handle exceptions. It retries a few times, then sends the message to a dead-letter topic. This prevents the consumer from stopping and helps you track bad messages.

java

import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.context.annotation.Bean;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.listener.DefaultErrorHandler;
import org.springframework.kafka.listener.DeadLetterPublishingRecoverer;
import org.springframework.kafka.support.ExponentialBackOffWithMaxRetries;
import org.springframework.stereotype.Component;

@Component
public class KafkaConfig {

    @Bean
    public DefaultErrorHandler errorHandler(KafkaTemplate<Object, Object> template) {
        DeadLetterPublishingRecoverer recoverer = new DeadLetterPublishingRecoverer(template);
        ExponentialBackOffWithMaxRetries backOff = new ExponentialBackOffWithMaxRetries(3);
        backOff.setInitialInterval(1000L);
        backOff.setMultiplier(2.0);
        backOff.setMaxInterval(10000L);
        return new DefaultErrorHandler(recoverer, backOff);
    }
}

Output

Consumer retries 3 times, then message sent to dead-letter topic 'my-topic.DLT' without stopping the consumer.

🛡️

Prevention

To avoid errors stopping your Kafka consumer, always configure an error handler like DefaultErrorHandler. Use dead-letter topics to isolate bad messages and monitor them. Validate messages before processing to reduce exceptions.

Use retries with backoff to handle temporary issues.
Log errors clearly for debugging.
Test your consumer logic with bad data.

⚠️

Related Errors

Common related errors include:

SerializationException: Happens if message format is wrong. Fix by checking serializers.
TimeoutException: Consumer can't connect to Kafka. Fix by checking network and broker status.
CommitFailedException: Happens if offsets can't be committed. Fix by adjusting consumer configs.

✅

Key Takeaways

Always configure a DefaultErrorHandler in Spring Kafka to manage consumer errors gracefully.

Use DeadLetterPublishingRecoverer to send failed messages to a dead-letter topic for later analysis.

Implement retries with backoff to handle temporary processing failures without stopping the consumer.

Validate and sanitize messages before processing to reduce runtime exceptions.

Monitor dead-letter topics and logs to quickly identify and fix message issues.