Shovel and Federation for multi-DC in RabbitMQ - Mini Project: Build & Apply

Shovel and Federation for multi-DC

📖 Scenario: You manage two data centers (DCs) that each run RabbitMQ servers. You want to move messages between these DCs reliably. This helps keep your systems in sync and share data across locations.

🎯 Goal: Set up RabbitMQ Shovel and Federation to transfer messages from one data center to another automatically.

📋 What You'll Learn

Create a source queue named source_queue on the first RabbitMQ server.

Configure a Shovel to move messages from source_queue to a destination queue dest_queue on the second RabbitMQ server.

Set up Federation upstream to link the two RabbitMQ servers.

Verify messages are transferred correctly by printing the message count on dest_queue.

💡 Why This Matters

🌍 Real World

Many companies run multiple data centers for reliability and speed. RabbitMQ Shovel and Federation help keep messages flowing between these centers automatically.

💼 Career

Understanding Shovel and Federation is important for DevOps engineers managing distributed messaging systems and ensuring data consistency across locations.

1

Create the source queue

Create a queue named source_queue on the first RabbitMQ server using the rabbitmqadmin command.

RabbitMQ

# Use rabbitmqadmin to declare the queue source_queue
# Your code here

Need a hint?

Use rabbitmqadmin declare queue name=source_queue durable=true to create the queue.

2

Configure the Shovel

Create a Shovel configuration named my_shovel that moves messages from source_queue on the first server to dest_queue on the second server. Use the AMQP URI amqp://user:password@second-server for the destination.

RabbitMQ

# Add shovel configuration named my_shovel with source source_queue and destination dest_queue
# Your code here

Need a hint?

Use rabbitmqctl set_parameter shovel my_shovel '{"src-uri":"amqp://","src-queue":"source_queue","dest-uri":"amqp://user:password@second-server","dest-queue":"dest_queue"}'.

3

Set up Federation upstream

Configure a Federation upstream named upstream_dc2 on the first RabbitMQ server to connect to the second server at amqp://user:password@second-server.

RabbitMQ

# Add federation upstream named upstream_dc2 pointing to amqp://user:password@second-server
# Your code here

Need a hint?

Use rabbitmqctl set_parameter federation-upstream upstream_dc2 '{"uri":"amqp://user:password@second-server"}'.

4

Verify message transfer

Print the message count on the dest_queue on the second RabbitMQ server using rabbitmqadmin.

RabbitMQ

# Use rabbitmqadmin to get message count on dest_queue
# Your code here

Need a hint?

Use rabbitmqadmin -H second-server -f value list queues name=dest_queue messages to see the message count.