Mirrored queues for redundancy in RabbitMQ - Commands & Configuration

Introduction

Sometimes messages in a queue can be lost if the server crashes. Mirrored queues copy messages to multiple servers to keep them safe. This way, if one server fails, another has the same messages ready.

When you want to make sure messages are not lost if one RabbitMQ server goes down

When running RabbitMQ in a cluster and you want queues to be available on multiple nodes

When your application depends on reliable message delivery even during server failures

When you want to balance load and have failover for message processing

When you need high availability for critical messaging systems

Config File - rabbitmq.conf

rabbitmq.conf

cluster_formation.peer_discovery_backend = rabbit_peer_discovery_classic_config

# Define policies for mirrored queues
# This policy mirrors all queues to all nodes in the cluster

policies = [{"ha-all", ".*", {"ha-mode", "all"}, {"priority", 0}}]

This configuration file sets up RabbitMQ to discover cluster nodes using classic config. The policies section defines a policy named ha-all that matches all queues (using .*) and sets ha-mode to all, which means every queue is mirrored to all nodes in the cluster. The priority sets the policy application order.

Commands

This command creates a policy named 'ha-all' that matches all queues and sets them to be mirrored on all nodes. It ensures redundancy for every queue.

Terminal

rabbitmqctl set_policy ha-all ".*" '{"ha-mode":"all"}'

Expected Output

Setting policy "ha-all" for pattern ".*" to "{"ha-mode":"all"}" ...

This command lists all policies currently set in RabbitMQ so you can verify the 'ha-all' policy is active.

Terminal

rabbitmqctl list_policies

Expected Output

Name    Pattern    Definition          Priority

ha-all  .*         {"ha-mode":"all"} 0

This command shows all queues and the policies applied to them, confirming that the mirrored queue policy is in effect.

Terminal

rabbitmqctl list_queues name policy

Expected Output

Name          Policy

my-queue      ha-all

another-queue ha-all

Key Concept

If you remember nothing else from this pattern, remember: Mirrored queues copy messages to multiple servers to keep them safe and available during failures.

Common Mistakes

Setting ha-mode to 'exactly' without specifying the number of nodes

RabbitMQ requires a number with 'exactly' mode, otherwise the policy is invalid and queues won't mirror.

Use 'ha-mode' set to 'all' for all nodes or 'exactly' with a number like {"ha-mode":"exactly", "ha-params":2}.

Not applying the policy before creating queues

Queues created before the policy is set will not be mirrored automatically.

Set the policy first, then create queues to ensure they are mirrored.

Using incorrect JSON syntax in the policy command

RabbitMQ rejects policies with invalid JSON, so the command fails silently or errors out.

Use single quotes around the JSON and double quotes inside, like '{"ha-mode":"all"}'.

Summary

Set a policy with rabbitmqctl to mirror queues across all nodes for redundancy.

Verify the policy is active using rabbitmqctl list_policies.

Check queues have the policy applied with rabbitmqctl list_queues.

Practice

1. What is the main purpose of using mirrored queues in RabbitMQ?

easy

A. To keep copies of messages on multiple servers for safety

B. To speed up message delivery by duplicating messages

C. To reduce the number of messages stored in the queue

D. To encrypt messages automatically during transmission

Solution

1. Step 1: Understand mirrored queues concept

   Mirrored queues replicate messages across multiple RabbitMQ nodes to ensure no message is lost if one node fails.

2. Step 2: Identify the main benefit

   This replication provides redundancy and safety, not speed or encryption.

3. Final Answer:

   To keep copies of messages on multiple servers for safety -> Option A

4. Quick Check:

   Mirroring = Redundancy and safety [OK]

Hint: Mirrored queues copy messages for safety, not speed or encryption [OK]

Common Mistakes:

• Thinking mirroring speeds up delivery
• Confusing mirroring with encryption
• Assuming mirroring reduces message count

2. Which command correctly sets a policy to mirror all queues in RabbitMQ?

easy

A. rabbitmqctl set_policy ha-all .* '{"ha-mode":"all"}'

B. rabbitmqctl set_policy ha-all ".*" '{"ha-mode":"all"}'

C. rabbitmqctl set_policy ha-all ".*" '{ha-mode:all}'

D. rabbitmqctl set_policy ha-all ".*" '{"ha-mode":all}'

Solution

1. Step 1: Review correct syntax for set_policy

   The command requires the policy name, pattern in quotes, and JSON string with keys and values in double quotes.

2. Step 2: Identify correct JSON format and quoting

   rabbitmqctl set_policy ha-all ".*" '{"ha-mode":"all"}' uses correct JSON with double quotes around keys and values and quotes the pattern ".*" properly.

3. Final Answer:

   rabbitmqctl set_policy ha-all ".*" '{"ha-mode":"all"}' -> Option B

4. Quick Check:

   Correct JSON and quoting = rabbitmqctl set_policy ha-all ".*" '{"ha-mode":"all"}' [OK]

Hint: Use double quotes inside JSON and quote pattern with ".*" [OK]

Common Mistakes:

• Missing quotes around JSON keys or values
• Not quoting the pattern string
• Using single quotes inside JSON instead of double quotes

3. Given this policy command:
rabbitmqctl set_policy ha-two ".*" '{"ha-mode":"exactly","ha-params":2}'
What will happen when a queue is created?

medium

A. The queue will be mirrored on one node only

B. The queue will be mirrored on all nodes

C. The queue will be mirrored on exactly two nodes

D. The queue will not be mirrored at all

Solution

1. Step 1: Understand ha-mode 'exactly' with ha-params

   The policy sets ha-mode to 'exactly' with ha-params 2, meaning the queue is mirrored on exactly 2 nodes.

2. Step 2: Apply policy effect on new queues

   Any queue matching ".*" will be mirrored on exactly two nodes as per the policy.

3. Final Answer:

   The queue will be mirrored on exactly two nodes -> Option C

4. Quick Check:

   ha-mode exactly + ha-params 2 = mirror on 2 nodes [OK]

Hint: ha-mode 'exactly' with ha-params sets mirror count [OK]

Common Mistakes:

• Confusing 'all' with 'exactly'
• Ignoring ha-params value
• Assuming no mirroring without ha-mode 'all'

4. You set this policy:
rabbitmqctl set_policy ha-all ".*" '{"ha-mode":"all"}'
But queues are not mirrored. What is the likely problem?

medium

A. The policy requires a restart of the queue or server to apply

B. The pattern ".*" does not match queue names

C. The policy was set but RabbitMQ was not restarted

D. The JSON syntax is incorrect

Solution

1. Step 1: Check policy application behavior

   Policies apply only to new queues or queues restarted after policy set.

2. Step 2: Identify why existing queues are not mirrored

   Existing queues created before policy need to be deleted or restarted to apply mirroring.

3. Final Answer:

   The policy requires a restart of the queue or server to apply -> Option A

4. Quick Check:

   Policy applies only on new or restarted queues [OK]

Hint: Policies apply only to new or restarted queues [OK]

Common Mistakes:

• Thinking RabbitMQ restart is needed
• Assuming pattern ".*" never matches
• Believing JSON syntax is wrong when it is correct

5. You want to ensure a queue named task_queue is mirrored on at least 3 nodes but no more than 5 using a range specification in ha-params. Which policy configuration achieves this?

hard

A. rabbitmqctl set_policy ha-range "^task_queue$" '{"ha-mode":"nodes","ha-params":["node1","node2","node3"]}'

B. rabbitmqctl set_policy ha-minmax ".*" '{"ha-mode":"exactly","ha-params":3}'

C. rabbitmqctl set_policy ha-minmax "^task_queue$" '{"ha-mode":"nodes","ha-params":["node1","node2","node3","node4","node5"]}'

D. rabbitmqctl set_policy ha-minmax "^task_queue$" '{"ha-mode":"exactly","ha-params":"3-5"}'

Solution

1. Step 1: Understand ha-mode 'exactly' with range ha-params

   ha-mode 'exactly' with ha-params as string like '3-5' dynamically mirrors the queue on a number of replicas between 3 and 5.

2. Step 2: Identify correct command

   rabbitmqctl set_policy ha-minmax "^task_queue$" '{"ha-mode":"exactly","ha-params":"3-5"}' matches the queue exactly and uses range specification.

3. Step 3: Check other options

   B uses fixed exactly 3 on all queues; C uses fixed nodes list of 5; D uses fixed nodes list of 3.

4. Final Answer:

   rabbitmqctl set_policy ha-minmax "^task_queue$" '{"ha-mode":"exactly","ha-params":"3-5"}' -> Option D

5. Quick Check:

   ha-mode exactly + ha-params "3-5" = dynamic range 3-5 [OK]

Hint: Use ha-mode 'exactly' with ha-params "3-5" for range [OK]

Common Mistakes:

• Using 'exactly' with fixed integer instead of range string
• Not matching queue name exactly
• Using 'nodes' mode with fixed list instead of range