How to Set Prefetch Count in RabbitMQ for Efficient Message Processing

To set the prefetch count in RabbitMQ, use the basic.qos method on the channel with the desired count as a parameter. This limits how many messages a consumer can receive before acknowledging previous ones, helping control load and improve processing efficiency.

Syntax

The basic.qos method sets the prefetch count on a channel. It takes an integer that defines how many messages the server will deliver to the consumer before waiting for acknowledgments.

• prefetch_count: Number of messages to prefetch.
• global: Boolean to apply setting to the entire channel (true) or per consumer (false).

python

channel.basic_qos(prefetch_count=10, global=False)

Example

This example shows how to set the prefetch count to 5 in a Python RabbitMQ consumer using the pika library. It limits the consumer to process 5 messages at a time before sending acknowledgments.

python

import pika

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()

# Set prefetch count to 5
channel.basic_qos(prefetch_count=5)

channel.queue_declare(queue='task_queue', durable=True)

def callback(ch, method, properties, body):
    print(f"Received {body.decode()}")
    # Simulate work
    import time
    time.sleep(1)
    print("Done")
    ch.basic_ack(delivery_tag=method.delivery_tag)

channel.basic_consume(queue='task_queue', on_message_callback=callback, auto_ack=False)

print('Waiting for messages. To exit press CTRL+C')
channel.start_consuming()

Output

Waiting for messages. To exit press CTRL+C
Received message1
Done
Received message2
Done
...

Common Pitfalls

• Setting prefetch_count too high can overload the consumer with many unacknowledged messages.
• Setting it too low may reduce throughput by limiting parallel processing.
• For multiple consumers on the same channel, use global=True carefully as it applies the limit to all consumers.
• Not acknowledging messages properly can cause message buildup despite prefetch settings.

python

## Wrong way: No prefetch set, consumer may get flooded
channel.basic_qos(prefetch_count=0)

## Right way: Limit to 10 messages
channel.basic_qos(prefetch_count=10)

Quick Reference

Parameter	Description	Default Value
prefetch_count	Max messages sent before ack	0 (unlimited)
global	Apply to channel (true) or consumer (false)	false

Key Takeaways

Use basic.qos with prefetch_count to control message flow to consumers.

Set prefetch_count to balance load and processing speed.

Use global parameter carefully when multiple consumers share a channel.

Always acknowledge messages to avoid blocking new deliveries.

Test different prefetch values to find the best for your workload.