Alerting on queue depth and consumer lag in RabbitMQ - Commands & Configuration

Introduction

Queues in RabbitMQ hold messages waiting to be processed. Sometimes, too many messages build up or consumers fall behind. Alerting on queue depth and consumer lag helps you fix problems before they affect your app.

When your app slows down because messages pile up in the queue.

When consumers stop processing messages and lag behind.

When you want to keep your message system healthy and responsive.

When you need to know if a queue is overloaded or stuck.

When you want to automate alerts instead of checking manually.

Config File - rabbitmq.conf

rabbitmq.conf

management.load_definitions = /etc/rabbitmq/definitions.json

# Enable Prometheus metrics for monitoring
prometheus.tcp.port = 9419

# Configure alert thresholds
alert.queue_depth_threshold = 1000
alert.consumer_lag_threshold = 500

This config enables management plugin definitions and Prometheus metrics for monitoring RabbitMQ.

The alert.queue_depth_threshold sets the max messages allowed in a queue before alerting.

The alert.consumer_lag_threshold sets the max lag allowed for consumers before alerting.

Commands

Enable the Prometheus plugin to expose RabbitMQ metrics for monitoring queue depth and consumer lag.

Terminal

rabbitmq-plugins enable rabbitmq_prometheus

Expected Output

The following plugins have been enabled:

rabbitmq_prometheus

Applying plugin configuration to rabbit@localhost... done.

List all queues with their current message count and number of consumers to check queue depth and consumer presence.

Terminal

rabbitmqctl list_queues name messages consumers

Expected Output

Listing queues ...

queue1	1200	2

queue2	300	1

queue3	0	0

...done.

Fetch queue details from RabbitMQ management API to see messages ready and consumers for alerting logic.

Terminal

curl -s http://localhost:15672/api/queues | jq '.[] | {name: .name, messages_ready: .messages_ready, consumers: .consumers}'

Expected Output

{

  "name": "queue1",

  "messages_ready": 1200,

  "consumers": 2

}

{

  "name": "queue2",

  "messages_ready": 300,

  "consumers": 1

}

{

  "name": "queue3",

  "messages_ready": 0,

  "consumers": 0

}

List consumers with their unacknowledged messages to detect consumer lag.

Terminal

rabbitmqctl list_consumers queue consumer_tag channel_pid ack_required unacked_messages

Expected Output

Listing consumers ...

queue1	consumer1	<0.1234.0>	true	600

queue1	consumer2	<0.1235.0>	true	700

queue2	consumer3	<0.1236.0>	true	100

...done.

Key Concept

If you remember nothing else from this pattern, remember: monitoring queue message counts and unacknowledged messages helps catch slow or stuck consumers early.

Common Mistakes

Checking only total messages without considering unacknowledged messages.

You might miss consumer lag because messages are delivered but not yet processed.

Check both messages_ready and unacknowledged messages to get full queue and consumer lag picture.

Not enabling the Prometheus plugin or management API before trying to monitor metrics.

No metrics or API data will be available, so alerts cannot be triggered.

Enable rabbitmq_prometheus plugin and management plugin to expose metrics and API.

Setting alert thresholds too high or too low without testing.

Alerts may trigger too often or never trigger, causing alert fatigue or missed problems.

Start with reasonable thresholds and adjust based on normal queue behavior.

Summary

Enable the Prometheus plugin to expose RabbitMQ metrics for monitoring.

Use rabbitmqctl and management API to check queue message counts and consumer lag.

Set alert thresholds to notify when queues grow too large or consumers fall behind.

Practice

1. What is the main reason to set up alerts on RabbitMQ queue depth?

easy

A. To increase the number of queues automatically

B. To delete messages faster from the queue

C. To catch message buildup early and prevent overload

D. To reduce the number of consumers

Solution

1. Step 1: Understand queue depth meaning

   Queue depth is the number of messages waiting in the queue.

2. Step 2: Identify alert purpose

   Alerting on queue depth helps detect when messages pile up, which can cause delays or failures.

3. Final Answer:

   To catch message buildup early and prevent overload -> Option C

4. Quick Check:

   Queue depth alert = catch buildup early [OK]

Hint: Queue depth alert means watching message pileup [OK]

Common Mistakes:

• Thinking alerts delete messages automatically
• Confusing queue depth with consumer count
• Assuming alerts increase queue size

2. Which RabbitMQ command correctly shows the current depth of a queue named task_queue?

easy

A. rabbitmqctl queue_status task_queue

B. rabbitmqctl show_queue task_queue depth

C. rabbitmqctl list_queues name messages

D. rabbitmqctl list_queues name messages_ready messages_unacknowledged

Solution

1. Step 1: Recall correct command syntax

   The command rabbitmqctl list_queues lists queues with specified columns.

2. Step 2: Identify columns showing queue depth

   Columns messages_ready and messages_unacknowledged together show total queue depth.

3. Final Answer:

   rabbitmqctl list_queues name messages_ready messages_unacknowledged -> Option D

4. Quick Check:

   List queues with messages_ready and unacknowledged = A [OK]

Hint: Use list_queues with messages_ready and messages_unacknowledged [OK]

Common Mistakes:

• Using non-existent commands like show_queue
• Omitting messages_unacknowledged count
• Confusing queue_status with list_queues

3. Given this output from rabbitmqctl list_queues name messages_ready messages_unacknowledged:

task_queue  50  10
logs_queue  0   0

What does this indicate about task_queue?

medium

A. There are 60 messages waiting or unacknowledged, indicating possible backlog

B. The queue is empty and consumers are caught up

C. There are 50 consumers connected to the queue

D. The queue is paused and not processing messages

Solution

1. Step 1: Add messages_ready and messages_unacknowledged

   50 + 10 = 60 total messages in or waiting acknowledgment.

2. Step 2: Interpret the meaning

   High number means messages are waiting or not yet processed, showing backlog.

3. Final Answer:

   There are 60 messages waiting or unacknowledged, indicating possible backlog -> Option A

4. Quick Check:

   messages_ready + messages_unacknowledged = backlog [OK]

Hint: Sum ready and unacknowledged messages for total queue depth [OK]

Common Mistakes:

• Confusing messages with consumer count
• Ignoring unacknowledged messages
• Assuming zero means backlog

4. You set an alert on consumer lag but it never triggers even when messages pile up. What is a likely cause?

medium

A. Consumers are processing messages but acknowledgments are delayed

B. The alert threshold is set too high to detect lag

C. The alert monitors only queue depth, not consumer lag

D. RabbitMQ does not support consumer lag monitoring

Solution

1. Step 1: Understand alert configuration

   If alert threshold is too high, small or moderate lag won't trigger alerts.

2. Step 2: Check alert settings

   Adjust threshold to a lower value to catch lag early.

3. Final Answer:

   The alert threshold is set too high to detect lag -> Option B

4. Quick Check:

   High threshold = no alert trigger [OK]

Hint: Check alert thresholds if alerts never trigger [OK]

Common Mistakes:

• Assuming RabbitMQ lacks lag monitoring
• Confusing queue depth with consumer lag
• Ignoring alert threshold settings

5. You want to create an alert that triggers when the task_queue has more than 100 messages waiting or unacknowledged, and consumer lag exceeds 30 seconds. Which approach is best?

hard

A. Use a monitoring tool to query messages_ready, messages_unacknowledged, and consumer lag metrics, then alert if thresholds exceeded

B. Set RabbitMQ to automatically delete messages after 30 seconds

C. Increase the number of consumers without monitoring queue depth

D. Manually check queue depth every hour and restart consumers if needed

Solution

1. Step 1: Identify metrics to monitor

   Queue depth is sum of messages_ready and messages_unacknowledged; consumer lag is time messages wait before processing.

2. Step 2: Use monitoring tool with alert rules

   Set alerts to trigger when queue depth > 100 and lag > 30 seconds for proactive response.

3. Final Answer:

   Use a monitoring tool to query messages_ready, messages_unacknowledged, and consumer lag metrics, then alert if thresholds exceeded -> Option A

4. Quick Check:

   Monitor metrics + alert on thresholds = C [OK]

Hint: Combine queue depth and lag metrics in monitoring alerts [OK]

Common Mistakes:

• Relying on manual checks instead of automated alerts
• Deleting messages automatically without monitoring
• Ignoring consumer lag in alerting