Exactly-once processing strategies in RabbitMQ - Commands & Configuration

Introduction

Sometimes messages get lost or processed multiple times in messaging systems. Exactly-once processing means each message is handled one time only, no more, no less. This avoids errors like duplicate orders or missed updates.

When you want to ensure a payment message is processed only once to avoid double charging.

When updating inventory counts where duplicates would cause wrong stock levels.

When sending notifications that must not be repeated to the same user.

When processing sensor data where repeated readings would skew results.

When coordinating distributed tasks that must not run multiple times.

Config File - rabbitmq.conf

rabbitmq.conf

loopback_users.guest = false

# Enable publisher confirms for reliable message delivery
channel_max = 2047

# Enable message acknowledgments
consumer_acknowledgement = true

# Enable persistent messages
queue_master_locator = min-masters

# Enable quorum queues for data safety
quorum_queues = true

This configuration disables guest loopback for security, enables publisher confirms to ensure messages are received by the broker, and requires consumers to acknowledge messages after processing. It also sets up quorum queues which replicate messages across nodes to prevent data loss, supporting exactly-once processing.

Commands

Create a virtual host to isolate your messaging environment.

Terminal

rabbitmqctl add_vhost my_vhost

Expected Output

No output (command runs silently)

Give user full permissions on the new virtual host to manage queues and exchanges.

Terminal

rabbitmqctl set_permissions -p my_vhost my_user ".*" ".*" ".*"

Expected Output

No output (command runs silently)

Create a durable quorum queue named task_queue to ensure message replication and persistence.

Terminal

rabbitmqadmin declare queue name=task_queue durable=true arguments='{"x-queue-type":"quorum"}'

Expected Output

{"queue":"task_queue","message_count":0,"consumer_count":0,"durable":true,"arguments":{"x-queue-type":"quorum"}}

Publish a persistent message to the task_queue with delivery_mode=2 to ensure it is saved to disk.

Terminal

rabbitmqadmin publish routing_key=task_queue payload='{"task":"process_order","order_id":12345}' properties='{"delivery_mode":2}'

Expected Output

{"routed":true}

Check the status of queues to verify the task_queue is durable and has messages waiting.

Terminal

rabbitmqctl list_queues name durable messages consumers

Expected Output

task_queue	true	1	0

Key Concept

If you remember nothing else from this pattern, remember: use durable quorum queues with publisher confirms and consumer acknowledgments to achieve exactly-once message processing.

Common Mistakes

Not setting the queue as durable or using classic queues instead of quorum queues.

Messages can be lost if the broker crashes or messages are not replicated.

Always declare queues as durable and use quorum queues for replication and safety.

Not enabling publisher confirms or ignoring them.

You won't know if the broker received the message, risking message loss.

Enable publisher confirms and wait for confirmation before considering the message sent.

Not acknowledging messages after processing.

Messages may be redelivered causing duplicates or lost if auto-ack is used incorrectly.

Use manual acknowledgments and only ack after successful processing.

Summary

Create durable quorum queues to replicate messages and prevent loss.

Use publisher confirms to ensure messages reach the broker safely.

Require consumers to manually acknowledge messages after processing.