Mirrored queues for redundancy in RabbitMQ - Step-by-Step Execution

Process Flow - Mirrored queues for redundancy

Client sends message

↓

Message arrives at Master Queue

↓

Master replicates message

↓

Messages copied to Mirror Queues

↓

Consumers read from Master or Mirror

↓

If Master fails -> Mirror promoted to Master

↓

Message flow continues without loss

Messages sent to the master queue are copied to mirror queues on other nodes. If the master fails, a mirror takes over to keep messages safe.

Execution Sample

RabbitMQ

rabbitmqctl set_policy ha-all "" '{"ha-mode":"all"}'
rabbitmqctl list_queues name policy

# Publish message to queue 'task_queue'
# Consumer reads from queue
# Master failure triggers mirror takeover

This setup command creates mirrored queues across all nodes for redundancy.

Process Table

Step	Action	Master Queue State	Mirror Queues State	Result
1	Create queue 'task_queue' with ha-mode=all	Empty	Empty on all mirrors	Queue created and mirrored
2	Publish message 'msg1' to master	Contains 'msg1'	Contains 'msg1' replicated	Message replicated to mirrors
3	Consumer reads 'msg1' from master	Empty	Contains 'msg1'	Message acknowledged on master
4	Master node fails	Unavailable	Contains 'msg1' on mirrors	Mirror promoted to master
5	Consumer reads 'msg1' from new master	Empty	Empty after ack	Message consumed without loss
6	Publish message 'msg2' to new master	Contains 'msg2'	Contains 'msg2' on mirrors	New messages replicated
7	Consumer reads 'msg2' from new master	Empty	Empty after ack	Message consumed
8	Master restored and rejoins cluster	Empty	Empty	Cluster stable with mirrored queues

💡 Execution stops after cluster stabilizes with mirrored queues ensuring no message loss despite master failure.

Status Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	After Step 5	After Step 6	After Step 7	Final
Master Queue Messages	Empty	['msg1']	Empty	Unavailable	Empty	['msg2']	Empty	Empty
Mirror Queues Messages	Empty	['msg1']	['msg1']	['msg1']	Empty	['msg2']	Empty	Empty
Master Node Status	Active	Active	Active	Failed	Failed	Active	Active	Restored
Mirror Node Status	Active	Active	Active	Active (promoted)	Active (promoted)	Active (promoted)	Active (promoted)	Active

Key Moments - 3 Insights

Why does the message still exist on mirror queues after the master queue message is acknowledged?

Because the master acknowledges the message first, but mirrors keep a copy until they also confirm consumption, ensuring no message loss if master fails (see step 3 and 5).

What happens to message flow when the master node fails?

The mirror queue is promoted to master and continues message delivery without interruption, as shown in step 4 and 5.

Why do we need to set the policy with ha-mode=all?

This policy tells RabbitMQ to mirror the queue on all nodes, ensuring redundancy and failover capability (step 1).

Visual Quiz - 3 Questions

Look at the execution table, what is the state of the master queue after step 3?

A Empty

B Contains 'msg1'

C Unavailable

D Contains 'msg2'

💡 Hint

Check the 'Master Queue State' column at step 3 in the execution table.

At which step does the mirror queue get promoted to master?

A Step 2

B Step 4

C Step 6

D Step 8

💡 Hint

Look for 'Mirror promoted to master' in the 'Result' column.

If the ha-mode policy was set to 'nodes' instead of 'all', how would the message replication change?

A Messages replicate to all nodes

B No replication happens

C Messages replicate only to specified nodes

D Messages replicate only to master

💡 Hint

Refer to the explanation in key moments about ha-mode policy effects.

Concept Snapshot

Mirrored queues replicate messages across nodes for redundancy.
Set policy with ha-mode=all to mirror on all nodes.
Master queue handles messages; mirrors keep copies.
If master fails, a mirror is promoted to master.
This ensures no message loss and high availability.

Full Transcript

Mirrored queues in RabbitMQ work by replicating messages from a master queue to mirror queues on other nodes. When a client sends a message, it arrives at the master queue and is copied to mirrors. Consumers read from the master queue normally. If the master node fails, one of the mirrors is promoted to master, allowing message consumption to continue without loss. This setup requires setting a policy with ha-mode=all to enable mirroring on all nodes. The execution trace shows queue states and message flow through publishing, consumption, failure, and recovery steps, illustrating how redundancy is maintained.

Practice

1. What is the main purpose of using mirrored queues in RabbitMQ?

easy

A. To keep copies of messages on multiple servers for safety

B. To speed up message delivery by duplicating messages

C. To reduce the number of messages stored in the queue

D. To encrypt messages automatically during transmission

Solution

1. Step 1: Understand mirrored queues concept

   Mirrored queues replicate messages across multiple RabbitMQ nodes to ensure no message is lost if one node fails.

2. Step 2: Identify the main benefit

   This replication provides redundancy and safety, not speed or encryption.

3. Final Answer:

   To keep copies of messages on multiple servers for safety -> Option A

4. Quick Check:

   Mirroring = Redundancy and safety [OK]

Hint: Mirrored queues copy messages for safety, not speed or encryption [OK]

Common Mistakes:

• Thinking mirroring speeds up delivery
• Confusing mirroring with encryption
• Assuming mirroring reduces message count

2. Which command correctly sets a policy to mirror all queues in RabbitMQ?

easy

A. rabbitmqctl set_policy ha-all .* '{"ha-mode":"all"}'

B. rabbitmqctl set_policy ha-all ".*" '{"ha-mode":"all"}'

C. rabbitmqctl set_policy ha-all ".*" '{ha-mode:all}'

D. rabbitmqctl set_policy ha-all ".*" '{"ha-mode":all}'

Solution

1. Step 1: Review correct syntax for set_policy

   The command requires the policy name, pattern in quotes, and JSON string with keys and values in double quotes.

2. Step 2: Identify correct JSON format and quoting

   rabbitmqctl set_policy ha-all ".*" '{"ha-mode":"all"}' uses correct JSON with double quotes around keys and values and quotes the pattern ".*" properly.

3. Final Answer:

   rabbitmqctl set_policy ha-all ".*" '{"ha-mode":"all"}' -> Option B

4. Quick Check:

   Correct JSON and quoting = rabbitmqctl set_policy ha-all ".*" '{"ha-mode":"all"}' [OK]

Hint: Use double quotes inside JSON and quote pattern with ".*" [OK]

Common Mistakes:

• Missing quotes around JSON keys or values
• Not quoting the pattern string
• Using single quotes inside JSON instead of double quotes

3. Given this policy command:
rabbitmqctl set_policy ha-two ".*" '{"ha-mode":"exactly","ha-params":2}'
What will happen when a queue is created?

medium

A. The queue will be mirrored on one node only

B. The queue will be mirrored on all nodes

C. The queue will be mirrored on exactly two nodes

D. The queue will not be mirrored at all

Solution

1. Step 1: Understand ha-mode 'exactly' with ha-params

   The policy sets ha-mode to 'exactly' with ha-params 2, meaning the queue is mirrored on exactly 2 nodes.

2. Step 2: Apply policy effect on new queues

   Any queue matching ".*" will be mirrored on exactly two nodes as per the policy.

3. Final Answer:

   The queue will be mirrored on exactly two nodes -> Option C

4. Quick Check:

   ha-mode exactly + ha-params 2 = mirror on 2 nodes [OK]

Hint: ha-mode 'exactly' with ha-params sets mirror count [OK]

Common Mistakes:

• Confusing 'all' with 'exactly'
• Ignoring ha-params value
• Assuming no mirroring without ha-mode 'all'

4. You set this policy:
rabbitmqctl set_policy ha-all ".*" '{"ha-mode":"all"}'
But queues are not mirrored. What is the likely problem?

medium

A. The policy requires a restart of the queue or server to apply

B. The pattern ".*" does not match queue names

C. The policy was set but RabbitMQ was not restarted

D. The JSON syntax is incorrect

Solution

1. Step 1: Check policy application behavior

   Policies apply only to new queues or queues restarted after policy set.

2. Step 2: Identify why existing queues are not mirrored

   Existing queues created before policy need to be deleted or restarted to apply mirroring.

3. Final Answer:

   The policy requires a restart of the queue or server to apply -> Option A

4. Quick Check:

   Policy applies only on new or restarted queues [OK]

Hint: Policies apply only to new or restarted queues [OK]

Common Mistakes:

• Thinking RabbitMQ restart is needed
• Assuming pattern ".*" never matches
• Believing JSON syntax is wrong when it is correct

5. You want to ensure a queue named task_queue is mirrored on at least 3 nodes but no more than 5 using a range specification in ha-params. Which policy configuration achieves this?

hard

A. rabbitmqctl set_policy ha-range "^task_queue$" '{"ha-mode":"nodes","ha-params":["node1","node2","node3"]}'

B. rabbitmqctl set_policy ha-minmax ".*" '{"ha-mode":"exactly","ha-params":3}'

C. rabbitmqctl set_policy ha-minmax "^task_queue$" '{"ha-mode":"nodes","ha-params":["node1","node2","node3","node4","node5"]}'

D. rabbitmqctl set_policy ha-minmax "^task_queue$" '{"ha-mode":"exactly","ha-params":"3-5"}'

Solution

1. Step 1: Understand ha-mode 'exactly' with range ha-params

   ha-mode 'exactly' with ha-params as string like '3-5' dynamically mirrors the queue on a number of replicas between 3 and 5.

2. Step 2: Identify correct command

   rabbitmqctl set_policy ha-minmax "^task_queue$" '{"ha-mode":"exactly","ha-params":"3-5"}' matches the queue exactly and uses range specification.

3. Step 3: Check other options

   B uses fixed exactly 3 on all queues; C uses fixed nodes list of 5; D uses fixed nodes list of 3.

4. Final Answer:

   rabbitmqctl set_policy ha-minmax "^task_queue$" '{"ha-mode":"exactly","ha-params":"3-5"}' -> Option D

5. Quick Check:

   ha-mode exactly + ha-params "3-5" = dynamic range 3-5 [OK]

Hint: Use ha-mode 'exactly' with ha-params "3-5" for range [OK]

Common Mistakes:

• Using 'exactly' with fixed integer instead of range string
• Not matching queue name exactly
• Using 'nodes' mode with fixed list instead of range