Queue length limits in RabbitMQ - Commands & Configuration

Introduction

Sometimes queues get too long and use too much memory. Queue length limits help stop queues from growing beyond a set size to keep the system stable.

When you want to prevent a queue from using too much memory by limiting how many messages it holds.

When you want to drop or reject new messages if the queue is full to avoid slowing down the system.

When you want to keep your RabbitMQ server responsive under heavy load by controlling queue size.

When you want to avoid message buildup that can cause delays in processing.

When you want to automatically remove old messages to make room for new ones.

Config File - queue-limit-policy.json

queue-limit-policy.json

{
  "vhost": "/",
  "name": "limit-queue-length",
  "pattern": "^my-queue$",
  "definition": {
    "max-length": 1000,
    "overflow": "reject-publish"
  },
  "priority": 0,
  "apply-to": "queues"
}

This JSON file defines a policy for RabbitMQ queues.

vhost: The virtual host where the policy applies.

name: The name of the policy.

pattern: Regex to match queue names this policy applies to.

definition: Contains max-length which limits queue size to 1000 messages, and overflow set to reject-publish which rejects new messages when full.

priority: Policy priority if multiple policies match.

apply-to: Specifies this policy applies to queues.

Commands

This command creates a policy named 'limit-queue-length' that limits queues named 'my-queue' to 1000 messages and rejects new messages when full.

Terminal

rabbitmqctl set_policy limit-queue-length '^my-queue$' '{"max-length":1000,"overflow":"reject-publish"}' --apply-to queues

Expected Output

Setting policy "limit-queue-length" for pattern "^my-queue$" to "{\"max-length\":1000,\"overflow\":\"reject-publish\"}" for queues in vhost "/" ...

→

--apply-to - Specifies the policy applies to queues

This command lists all queues with their names and how many messages are ready or unacknowledged, so you can check queue length.

Terminal

rabbitmqctl list_queues name messages_ready messages_unacknowledged

Expected Output

Listing queues ...

my-queue	500	0

another-queue	10	2

This command removes the 'limit-queue-length' policy if you want to stop limiting the queue size.

Terminal

rabbitmqctl clear_policy limit-queue-length

Expected Output

Clearing policy "limit-queue-length" ...

Key Concept

If you remember nothing else from this pattern, remember: setting a max-length with overflow controls how many messages a queue holds and what happens when it is full.

Common Mistakes

Not using the correct regex pattern to match the queue name.

The policy won't apply if the pattern does not match the queue name exactly.

Use '^queue-name$' to match the exact queue name or a proper regex to match multiple queues.

Forgetting to specify --apply-to queues flag.

The policy may not apply to queues and will be ignored or applied incorrectly.

Always include --apply-to queues when setting queue policies.

Setting max-length too low without considering message flow.

Messages get rejected too often, causing message loss or errors.

Choose a max-length that balances memory use and message throughput.

Summary

Use rabbitmqctl set_policy with max-length and overflow to limit queue size and control message behavior when full.

Check queue lengths with rabbitmqctl list_queues to monitor message counts.

Remove policies with rabbitmqctl clear_policy when limits are no longer needed.

Practice

1. What is the main purpose of setting a max-length limit on a RabbitMQ queue?

easy

A. To encrypt messages in the queue

B. To increase the speed of message delivery

C. To limit the number of messages the queue can hold to prevent overload

D. To allow unlimited messages in the queue

Solution

1. Step 1: Understand queue length limits

   Queue length limits control how many messages a queue can hold to avoid memory overload.

2. Step 2: Identify the purpose of max-length

   The max-length setting limits the number of messages to prevent backlog and resource exhaustion.

3. Final Answer:

   To limit the number of messages the queue can hold to prevent overload -> Option C

4. Quick Check:

   Queue length limits = prevent overload [OK]

Hint: Queue length limits prevent too many messages piling up [OK]

Common Mistakes:

• Thinking max-length speeds up delivery
• Confusing max-length with encryption
• Assuming queues can hold unlimited messages

2. Which of the following is the correct syntax to set a queue length limit of 1000 messages using rabbitmqctl?

easy

A. rabbitmqctl set_policy mypolicy "^myqueue$" '{"max-length":1000}' --apply-to queues

B. rabbitmqctl set_policy mypolicy "myqueue" '{max-length=1000}' --apply-to queues

C. rabbitmqctl set_policy mypolicy "^myqueue$" '{max-length:1000}' --apply-to queues

D. rabbitmqctl set_policy mypolicy "^myqueue$" '{"max-length"=1000}' --apply-to queues

Solution

1. Step 1: Review correct JSON syntax for policy

   The policy value must be a JSON string with double quotes around keys and values, e.g., '{"max-length":1000}'.

2. Step 2: Check the full command syntax

   The correct command uses regex pattern with ^ and $ for exact queue name, and --apply-to queues to target queues.

3. Final Answer:

   rabbitmqctl set_policy mypolicy "^myqueue$" '{"max-length":1000}' --apply-to queues -> Option A

4. Quick Check:

   Correct JSON and command syntax = rabbitmqctl set_policy mypolicy "^myqueue$" '{"max-length":1000}' --apply-to queues [OK]

Hint: Use double quotes and JSON format for policy values [OK]

Common Mistakes:

• Using single quotes inside JSON incorrectly
• Missing regex anchors ^ and $
• Using = instead of : in JSON

3. Given the policy set with rabbitmqctl set_policy limit1000 "^task_queue$" '{"max-length":1000}' --apply-to queues, what happens when the queue task_queue receives its 1001st message?

medium

A. The new message is rejected and not added to the queue

B. The oldest message in the queue is dropped to make room for the new one

C. The queue size automatically increases to hold the new message

D. RabbitMQ crashes due to overflow

Solution

1. Step 1: Understand max-length behavior

   When max-length is reached, RabbitMQ drops oldest messages to keep queue size within limit.

2. Step 2: Apply to the 1001st message

   On the 1001st message, the oldest message is removed to add the new one, maintaining max 1000 messages.

3. Final Answer:

   The oldest message in the queue is dropped to make room for the new one -> Option B

4. Quick Check:

   max-length reached = oldest message dropped [OK]

Hint: max-length drops oldest message, never rejects new [OK]

Common Mistakes:

• Thinking new messages get rejected
• Assuming queue size auto-expands
• Believing RabbitMQ crashes on overflow

4. You tried to set a queue length limit with this command: rabbitmqctl set_policy limit "^queue$" '{max-length:500}' --apply-to queues but it did not work. What is the error?

medium

A. The JSON policy value keys and values must be double quoted strings

B. The JSON policy value uses colon instead of equal sign

C. The JSON policy value uses single quotes instead of double quotes

D. The regex pattern is missing anchors ^ and $

Solution

1. Step 1: Check JSON syntax in policy

   JSON requires keys and string values to be double quoted, e.g., '{"max-length":500}'.

2. Step 2: Identify the error in given command

   The command uses '{max-length:500}' without quotes around key, which is invalid JSON.

3. Final Answer:

   The JSON policy value keys and values must be double quoted strings -> Option A

4. Quick Check:

   JSON keys/values need double quotes [OK]

Hint: Always double quote keys and string values in JSON [OK]

Common Mistakes:

• Using single quotes inside JSON
• Using colon instead of colon (correct is colon)
• Ignoring regex anchors (not error here)

5. You want to limit a queue named logs to 500 messages but also want to drop messages older than 1 hour. Which policy configuration correctly applies both limits?

hard

A. rabbitmqctl set_policy loglimit "^logs$" '{"max-length":500, "max-age":60}' --apply-to queues

B. rabbitmqctl set_policy loglimit "^logs$" '{"max-length":500, "max-age":3600}' --apply-to queues

C. rabbitmqctl set_policy loglimit "^logs$" '{"max-length":500, "max-age":60000}' --apply-to queues

D. rabbitmqctl set_policy loglimit "^logs$" '{"max-length":500, "max-age":3600000}' --apply-to queues

Solution

1. Step 1: Understand max-age unit

   RabbitMQ's max-age is in milliseconds, so 1 hour = 3600000 ms.

2. Step 2: Check policy JSON for both limits

   The policy must include both "max-length":500 and "max-age":3600000 correctly formatted.

3. Final Answer:

   rabbitmqctl set_policy loglimit "^logs$" '{"max-length":500, "max-age":3600000}' --apply-to queues -> Option D

4. Quick Check:

   1 hour = 3600000 ms, both limits set [OK]

Hint: Remember max-age is milliseconds, 1 hour = 3600000 [OK]

Common Mistakes:

• Using seconds or minutes instead of milliseconds
• Missing one of the limits in policy
• Incorrect JSON formatting