How to Use Fair Dispatch in RabbitMQ for Balanced Message Processing

To use fair dispatch in RabbitMQ, set the prefetch_count to 1 on the consumer channel using basic_qos. This ensures each consumer processes one message at a time, preventing message overload and balancing the load fairly.

Syntax

The key to fair dispatch in RabbitMQ is the basic_qos method with the prefetch_count parameter. Setting prefetch_count=1 tells RabbitMQ to send only one unacknowledged message at a time to each consumer.

Parts explained:

• channel.basic_qos(prefetch_count=1): Limits the number of unacknowledged messages per consumer to 1.
• channel.basic_consume(queue, callback, auto_ack=False): Consumes messages with manual acknowledgments.
• channel.basic_ack(delivery_tag): Acknowledges message processing completion.

python

channel.basic_qos(prefetch_count=1)
channel.basic_consume(queue='task_queue', on_message_callback=callback, auto_ack=False)

# Inside callback function:
ch.basic_ack(delivery_tag=method.delivery_tag)

Example

This example shows a Python consumer using pika library that implements fair dispatch by setting prefetch_count=1. It processes one message at a time and acknowledges it after processing.

python

import pika
import time

def callback(ch, method, properties, body):
    print(f"Received {body.decode()}")
    time.sleep(body.count(b'.'))  # Simulate work
    print("Done")
    ch.basic_ack(delivery_tag=method.delivery_tag)

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
channel.queue_declare(queue='task_queue', durable=True)

channel.basic_qos(prefetch_count=1)
channel.basic_consume(queue='task_queue', on_message_callback=callback, auto_ack=False)

print('Waiting for messages. To exit press CTRL+C')
channel.start_consuming()

Output

Waiting for messages. To exit press CTRL+C
Received Task 1...
Done
Received Task 2..
Done

Common Pitfalls

Common mistakes when using fair dispatch include:

• Not setting auto_ack=False, which disables manual acknowledgments and breaks fair dispatch.
• Not calling basic_ack after processing, causing messages to remain unacknowledged and blocking new messages.
• Setting prefetch_count too high, which defeats the purpose of fair dispatch.

Always combine prefetch_count=1 with manual acknowledgments to ensure fair load distribution.

python

## Wrong way (auto_ack=True disables manual ack):
channel.basic_consume(queue='task_queue', on_message_callback=callback, auto_ack=True)

## Right way:
channel.basic_qos(prefetch_count=1)
channel.basic_consume(queue='task_queue', on_message_callback=callback, auto_ack=False)

Quick Reference

Summary tips for fair dispatch in RabbitMQ:

• Use channel.basic_qos(prefetch_count=1) to limit unacknowledged messages per consumer.
• Set auto_ack=False to enable manual acknowledgments.
• Call channel.basic_ack() after processing each message.
• Declare queues as durable for reliability.

Key Takeaways

Set prefetch_count=1 with basic_qos to enable fair dispatch in RabbitMQ.

Always use manual acknowledgments (auto_ack=False) to control message flow.

Acknowledge each message after processing to allow new messages to be delivered.

Fair dispatch balances load by preventing consumers from being overwhelmed.

Declare queues as durable to ensure message persistence.