Consumer lag monitoring in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

💻 Command Output

intermediate

Check consumer lag using kafka-consumer-groups.sh

You run the command kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-group. What output will you see regarding consumer lag?

Kafka

kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-group

A

GROUP           TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                    HOST            CLIENT-ID
my-group        my-topic        0          150             100             -50             consumer-1-12345                               /127.0.0.1      consumer-1

B Error: Unknown group 'my-group' not found

C

GROUP           TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                    HOST            CLIENT-ID
my-group        my-topic        0          100             150             50              consumer-1-12345                               /127.0.0.1      consumer-1

D

GROUP           TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                    HOST            CLIENT-ID
my-group        my-topic        0          0               0               0               consumer-1-12345                               /127.0.0.1      consumer-1

💡 Hint

Lag is the difference between log end offset and current offset.

The lag is calculated as LOG-END-OFFSET minus CURRENT-OFFSET. It cannot be negative. Option C shows a positive lag of 50, which is correct. Option C shows negative lag, which is invalid. Option C is an error message, and option C shows zero lag.

🧠 Conceptual

intermediate

Understanding consumer lag causes

Which of the following is the most common cause of increasing consumer lag in Kafka?

A The consumer group ID is changed frequently

B The Kafka broker is down

C The topic has no partitions

D The consumer is processing messages slower than the producer is producing them

💡 Hint

Lag grows when consumption speed is less than production speed.

Consumer lag increases when the consumer cannot keep up with the rate of new messages produced. Broker downtime or topic partitions affect availability but do not directly cause lag to increase steadily. Changing group ID resets offsets but does not cause lag to grow.

🔀 Workflow

advanced

Steps to monitor consumer lag with Prometheus and Kafka Exporter

You want to monitor Kafka consumer lag using Prometheus and Kafka Exporter. Which sequence of steps is correct?

A 1,2,3,4

B 2,1,4,3

C 4,3,2,1

D 1,3,2,4

💡 Hint

Start by collecting metrics, then scrape, visualize, and alert.

First, Kafka Exporter collects metrics. Then Prometheus scrapes those metrics. Next, Grafana visualizes the data. Finally, alert rules notify on high lag.

❓ Troubleshoot

advanced

Troubleshoot missing consumer lag metrics in Prometheus

You notice Prometheus does not show consumer lag metrics after setting up Kafka Exporter. What is the most likely cause?

A Kafka Exporter is not configured with the correct Kafka bootstrap servers

B Prometheus is scraping metrics too frequently

C Grafana dashboard is not refreshed

D Kafka topic has zero partitions

💡 Hint

Check Kafka Exporter connection settings first.

If Kafka Exporter cannot connect to Kafka due to wrong bootstrap servers, it cannot collect metrics. Scraping frequency or Grafana refresh do not affect metric availability in Prometheus. Topic partitions do not prevent metrics collection.

✅ Best Practice

expert

Best practice for alerting on consumer lag

Which alerting strategy is best to avoid false alarms when monitoring Kafka consumer lag?

A Alert immediately when lag is greater than zero

B Alert only if lag exceeds a threshold for a sustained period

C Alert based on consumer group ID changes

D Alert only during business hours regardless of lag

💡 Hint

Consider lag spikes that resolve quickly.

Lag can spike briefly during normal operation. Alerting only after lag exceeds a threshold for some time reduces noise. Immediate alerts or alerts based on group ID changes cause many false alarms. Limiting alerts to business hours may miss critical issues.