You want to alert if consumer lag exceeds 1000 messages for any partition. Which approach is best?
hard📝 Workflow Q8 of Q15
Kafka - Monitoring and Operations
You want to alert if consumer lag exceeds 1000 messages for any partition. Which approach is best?
AUse a monitoring tool to periodically run kafka-consumer-groups.sh and parse lag values
BManually check lag once a day using kafka-topics.sh
CRestart consumers daily to reset lag
DIncrease the number of partitions to reduce lag
Step-by-Step Solution
Solution:
Step 1: Identify automated monitoring needs
To alert on lag thresholds, automation is needed to check lag regularly and trigger alerts.
Step 2: Evaluate options
Running kafka-consumer-groups.sh periodically and parsing output fits this need; manual checks are unreliable; restarting consumers does not fix lag causes; increasing partitions affects throughput but not direct lag alerting.
Final Answer:
Use a monitoring tool to periodically run kafka-consumer-groups.sh and parse lag values -> Option A
Quick Check:
Automate lag checks with kafka-consumer-groups.sh [OK]
Quick Trick:Automate lag checks with scripts or monitoring tools [OK]
Common Mistakes:
MISTAKES
Relying on manual lag checks
Restarting consumers without fixing lag cause
Changing partitions instead of monitoring
Master "Monitoring and Operations" in Kafka
9 interactive learning modes - each teaches the same concept differently