Fair dispatch with prefetch in RabbitMQ - Commands & Configuration

Introduction

When multiple workers consume messages from a queue, some may get overloaded while others stay idle. Fair dispatch with prefetch helps balance the load by limiting how many messages a worker can receive before acknowledging them.

When you have several workers processing tasks from the same queue and want to avoid one worker getting too many tasks at once.

When tasks vary in processing time and you want to prevent slow workers from blocking others.

When you want to improve resource usage by distributing work evenly across workers.

When you want to ensure that a worker only gets a new message after finishing the previous one.

When you want to avoid message loss by ensuring messages are acknowledged before sending more.

Commands

Check existing queues to confirm the queue you want to use exists.

Terminal

rabbitmqctl list_queues

Expected Output

Listing queues ...

my-task-queue	0

Create a durable queue named 'my-task-queue' to hold tasks persistently.

Terminal

rabbitmqadmin declare queue name=my-task-queue durable=true

Expected Output

Queue 'my-task-queue' declared

Send a task message 'Task 1' to the queue for workers to process.

Terminal

rabbitmqadmin publish routing_key=my-task-queue payload='Task 1'

Expected Output

Published message to queue 'my-task-queue'

Set prefetch count to 1 so each worker receives only one unacknowledged message at a time, enabling fair dispatch.

Terminal

rabbitmqctl set_prefetch_count 1

Expected Output

No output (command runs silently)

→

set_prefetch_count - Limits the number of unacknowledged messages per consumer

Verify that consumers are connected and prefetch settings are applied.

Terminal

rabbitmqctl list_consumers

Expected Output

Listing consumers ...

consumer_tag	my-task-queue	channel_details

Key Concept

If you remember nothing else from this pattern, remember: setting prefetch to 1 ensures each worker gets one message at a time, balancing the load fairly.

Common Mistakes

Not setting prefetch count, so workers get many messages at once.

This causes some workers to be overloaded while others are idle, leading to inefficient processing.

Always set prefetch count to 1 for fair dispatch when tasks vary in processing time.

Not acknowledging messages after processing.

Unacknowledged messages stay assigned to a worker, blocking new messages and causing delays.

Make sure each worker sends an acknowledgment after finishing a task.

Summary

Create a durable queue to hold tasks persistently.

Publish messages to the queue for workers to process.

Set prefetch count to 1 to ensure fair dispatch of messages.

Verify consumers and prefetch settings to confirm balanced load.