How to Fix Kafka Offset Out of Range Error Quickly

The offset out of range error in Kafka happens when a consumer tries to read a message offset that no longer exists on the broker. To fix it, reset the consumer's offset to a valid position using Kafka's --reset-offsets tool or configure the consumer to start from the earliest or latest offset.

Why This Happens

This error occurs because Kafka stores messages for a limited time or size. If a consumer tries to read an offset that was deleted due to log retention policies, Kafka throws an offset out of range error. This usually happens when the consumer is too slow or has been offline for a long time.

bash

kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic my-topic --partition 0 --offset 1000

Output

ERROR OffsetOutOfRangeCode: Offset out of range for partition my-topic-0. The earliest offset is 1050 and the latest offset is 1100.

The Fix

Reset the consumer offset to a valid position using Kafka's kafka-consumer-groups.sh --reset-offsets command. You can set the offset to the earliest or latest available offset to resume consumption without errors.

bash

kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group my-group --topic my-topic --reset-offsets --to-earliest --execute

Output

Offsets reset to earliest for group my-group on topic my-topic.

Prevention

To avoid this error, keep your consumers active and process messages promptly. Configure your consumer with auto.offset.reset=earliest or auto.offset.reset=latest to handle missing offsets gracefully. Also, monitor your Kafka retention policies and consumer lag regularly.

Related Errors

• Consumer Lag Too High: Causes delayed processing and potential offset errors.
• Group Coordinator Not Available: Consumer cannot commit offsets properly.
• Offset Commit Failed: Happens if consumer loses connection during commit.

Key Takeaways

The offset out of range error means the consumer offset is no longer valid on the broker.

Use kafka-consumer-groups.sh with --reset-offsets to fix the offset position.

Configure consumers with auto.offset.reset to handle missing offsets automatically.

Keep consumers running regularly to avoid lag and offset expiration.

Monitor Kafka retention and consumer lag to prevent this error.