Consumer prefetch optimization in RabbitMQ - Mini Project: Build & Apply

Consumer Prefetch Optimization with RabbitMQ

📖 Scenario: You are working with RabbitMQ to process messages efficiently. To avoid overwhelming a consumer with too many messages at once, you want to control how many messages the consumer fetches before acknowledging them. This is called prefetch count.Setting the right prefetch count helps balance load and improves system responsiveness.

🎯 Goal: Learn how to set the consumer prefetch count in RabbitMQ using Python's pika library. You will create a connection, configure prefetch count, consume messages, and print them.

📋 What You'll Learn

Create a RabbitMQ connection and channel

Set the prefetch count to 5 for the consumer

Consume messages from a queue named task_queue

Print each received message body

💡 Why This Matters

🌍 Real World

In real systems, controlling prefetch count helps prevent consumers from being overloaded and ensures fair message distribution.

💼 Career

Understanding consumer prefetch optimization is important for roles involving message queue management, backend development, and system reliability engineering.

1

Create RabbitMQ connection and channel

Create a RabbitMQ connection using pika.BlockingConnection with default parameters. Then create a channel from this connection and assign it to the variable channel.

RabbitMQ

# Create connection and channel
# Your code here

Need a hint?

Use pika.ConnectionParameters('localhost') to connect to the local RabbitMQ server.

2

Set consumer prefetch count

Use the channel.basic_qos method to set the prefetch count to 5. This limits the number of unacknowledged messages the consumer can receive at once.

RabbitMQ

import pika

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
# Set prefetch count to 5
# Your code here

Need a hint?

Call channel.basic_qos(prefetch_count=5) to set the limit.

3

Consume messages from task_queue

Define a callback function named callback that takes ch, method, properties, and body as parameters. Inside it, print the message body decoded as UTF-8. Then use channel.basic_consume to consume messages from the queue named task_queue with callback as the callback function and auto_ack=True.

RabbitMQ

import pika

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
channel.basic_qos(prefetch_count=5)

# Define callback and start consuming
# Your code here

Need a hint?

Define the callback function first, then call channel.basic_consume with the correct parameters.

4

Start consuming and print messages

Call channel.start_consuming() to start the consumer loop and print messages as they arrive.

RabbitMQ

import pika

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
channel.basic_qos(prefetch_count=5)

def callback(ch, method, properties, body):
    print(body.decode('utf-8'))

channel.basic_consume(queue='task_queue', on_message_callback=callback, auto_ack=True)

# Start consuming messages
# Your code here

Need a hint?

Call channel.start_consuming() to begin receiving messages.