How to Check Consumer Lag in Kafka: Simple Commands and Examples

To check consumer lag in Kafka, use the kafka-consumer-groups.sh command with the --describe option specifying your consumer group and bootstrap server. This shows the difference between the latest offset and the consumer's current offset, indicating lag.

Syntax

The basic syntax to check consumer lag is:

• kafka-consumer-groups.sh --bootstrap-server <broker-address> --describe --group <consumer-group>

Explanation:

• --bootstrap-server: Kafka broker address to connect.
• --describe: Shows detailed info about the consumer group.
• --group: The name of the consumer group you want to check.

bash

kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-consumer-group

Example

This example shows how to check lag for a consumer group named my-consumer-group connected to a Kafka broker at localhost:9092. The output lists each partition's current offset, log end offset, and lag.

bash

kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-consumer-group

Output

TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                    HOST            CLIENT-ID
my-topic       0          105             110             5               consumer-1-12345-67890                        /127.0.0.1      consumer-1
my-topic       1          200             200             0               consumer-2-12345-67890                        /127.0.0.1      consumer-2

Common Pitfalls

Common mistakes when checking consumer lag include:

• Using --zookeeper option with newer Kafka versions (2.4+), which is deprecated. Always use --bootstrap-server.
• Checking lag for the wrong consumer group name.
• Not having the correct permissions or network access to the Kafka broker.
• Confusing CURRENT-OFFSET with LOG-END-OFFSET. Lag is the difference between these two.

bash

Wrong (deprecated):
kafka-consumer-groups.sh --zookeeper localhost:2181 --describe --group my-consumer-group

Right (modern):
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-consumer-group

Quick Reference

Option	Description
--bootstrap-server	Kafka broker address to connect
--describe	Show detailed consumer group info
--group	Specify the consumer group to check
CURRENT-OFFSET	Offset where consumer currently is
LOG-END-OFFSET	Latest offset in the partition
LAG	Difference between log end and current offset

Key Takeaways

Use kafka-consumer-groups.sh with --bootstrap-server and --describe to check consumer lag.

Lag is the difference between LOG-END-OFFSET and CURRENT-OFFSET for each partition.

Avoid using deprecated --zookeeper option; use --bootstrap-server instead.

Ensure you specify the correct consumer group name to get accurate lag info.

Check network access and permissions to connect to Kafka brokers.