Fair dispatch with prefetch in RabbitMQ - Step-by-Step Execution

Process Flow - Fair dispatch with prefetch

Consumer connects to queue

↓

Set prefetch count = N

↓

Receive up to N messages

↓

Process message

↓

Send ack to server

↓

Server sends next message

↩Back to Receive

The consumer sets a limit (prefetch count) on how many messages it can handle at once. The server sends only that many messages before waiting for acknowledgments, ensuring fair load distribution.

Execution Sample

RabbitMQ

channel.basic_qos(prefetch_count=1)
channel.basic_consume(queue='task_queue', on_message_callback=callback)
channel.start_consuming()

This code sets the prefetch count to 1, so the consumer gets one message at a time and acknowledges it before receiving the next.

Process Table

Step	Prefetch Count	Messages Sent by Server	Messages Processed by Consumer	Acknowledgments Sent	Server Waits?
1	1	1	0	0	No
2	1	1	1	1	No
3	1	1	1	1	Yes
4	1	2	2	2	No
5	1	2	2	2	Yes
6	1	3	3	3	No
7	1	3	3	3	Yes
8	1	4	4	4	No
9	1	4	4	4	Yes
10	1	5	5	5	No
Exit	1	5	5	5	Yes - no more messages

💡 All messages processed and acknowledged; server has no more messages to send.

Status Tracker

Variable	Start	After 1	After 2	After 3	After 4	After 5	Final
Messages Sent by Server	0	1	1	2	2	3	5
Messages Processed by Consumer	0	0	1	1	2	3	5
Acknowledgments Sent	0	0	1	1	2	3	5

Key Moments - 3 Insights

Why does the server send only one message at a time even if more are waiting?

Because the prefetch count is set to 1, the server waits for an acknowledgment before sending the next message, ensuring fair dispatch as shown in steps 1 and 2 of the execution_table.

What happens if the consumer does not acknowledge a message?

The server will not send more messages beyond the prefetch count, causing it to wait indefinitely as the acknowledgment is required to proceed, demonstrated by the 'Server Waits?' column.

Why is fair dispatch important in message processing?

It prevents one consumer from being overloaded by limiting unacknowledged messages, ensuring all consumers get a fair share of work, as seen by the controlled message flow in the execution_table.

Visual Quiz - 3 Questions

Look at the execution_table at step 4, how many messages has the consumer processed?

A 2

B 1

C 3

D 0

💡 Hint

Check the 'Messages Processed by Consumer' column at step 4 in the execution_table.

At which step does the server stop sending messages because all are processed?

A Step 10

B Exit

C Step 8

D Step 5

💡 Hint

Look at the 'Messages Sent by Server' column and the exit_note in the execution_table.

If prefetch_count was set to 2, how would the 'Messages Sent by Server' change at step 1?

A It would be 1

B It would be 0

C It would be 2

D It would be 3

💡 Hint

Prefetch count controls how many messages the server sends before waiting for ack; see concept_flow.

Concept Snapshot

Fair dispatch with prefetch:
- Set prefetch_count to limit unacknowledged messages.
- Server sends only up to prefetch_count messages.
- Consumer processes and acknowledges each message.
- Server waits for ack before sending more.
- Ensures balanced load among consumers.

Full Transcript

Fair dispatch with prefetch in RabbitMQ means the consumer tells the server how many messages it can handle at once by setting prefetch_count. The server then sends only that many messages before waiting for acknowledgments. This prevents one consumer from getting overloaded and ensures messages are fairly distributed. The example code sets prefetch_count to 1, so the consumer gets one message at a time. The execution table shows the server sending one message, the consumer processing and acknowledging it, then the server sending the next. This cycle repeats until all messages are processed. If the consumer does not acknowledge, the server stops sending new messages. This method balances work fairly among consumers.