Work queue for task distribution in RabbitMQ - Mini Project: Build & Apply

Work queue for task distribution

📖 Scenario: You are building a simple task distribution system using RabbitMQ. Multiple workers will receive tasks from a queue to share the workload evenly.

🎯 Goal: Create a RabbitMQ work queue named task_queue. Write a producer script that sends 5 tasks with specific messages to the queue. Then write a consumer script that receives and prints each task message.

📋 What You'll Learn

Create a durable queue named task_queue

Producer sends 5 messages: Task 1, Task 2, Task 3, Task 4, Task 5

Consumer receives messages from task_queue and prints them

Use manual message acknowledgments in the consumer

💡 Why This Matters

🌍 Real World

Work queues are used in real systems to distribute tasks like image processing, email sending, or data analysis among multiple worker machines.

💼 Career

Understanding work queues and message brokers like RabbitMQ is essential for DevOps engineers and backend developers to build scalable and reliable distributed systems.

1

Create a durable queue named task_queue

Write a Python script that connects to RabbitMQ on localhost and declares a durable queue named task_queue. Use the pika library and set durable=True when declaring the queue.

RabbitMQ

# Your code here to connect and declare the durable queue 'task_queue'

Hint

Use pika.BlockingConnection to connect and channel.queue_declare with durable=True.

2

Send 5 tasks to the task_queue

Add code to the script to send 5 messages: 'Task 1', 'Task 2', 'Task 3', 'Task 4', and 'Task 5' to the task_queue. Make sure to mark messages as persistent by setting delivery_mode=2 in BasicProperties.

RabbitMQ

import pika

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
channel.queue_declare(queue='task_queue', durable=True)

# Your code here to send 5 persistent messages 'Task 1' to 'Task 5'

connection.close()

Hint

Use a for loop from 1 to 5 and channel.basic_publish with delivery_mode=2 for persistence.

3

Create a consumer to receive and print tasks

Write a Python script that connects to RabbitMQ, declares the same durable queue task_queue, and consumes messages. Use channel.basic_consume with a callback function that prints the received message body. Use manual acknowledgments by calling ch.basic_ack inside the callback.

RabbitMQ

# Your code here to connect, declare queue, and consume messages with manual ack

Hint

Define a callback function that prints the message and calls basic_ack. Use channel.basic_consume with this callback.

4

Run the consumer and verify output

Run the consumer script to receive and print all 5 task messages sent earlier. The output should show:
Received Task 1
Received Task 2
Received Task 3
Received Task 4
Received Task 5

RabbitMQ

import pika

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
channel.queue_declare(queue='task_queue', durable=True)

def callback(ch, method, properties, body):
    print(f"Received {body.decode()}")
    ch.basic_ack(delivery_tag=method.delivery_tag)

channel.basic_consume(queue='task_queue', on_message_callback=callback)

print('Waiting for messages. To exit press CTRL+C')
channel.start_consuming()

# Run this script and observe the printed output

Hint

Run the consumer script after the producer has sent messages. The output should print each task received.

Practice

1. What is the main purpose of a work queue in RabbitMQ?

easy

A. To create a backup of messages

B. To distribute tasks among multiple workers safely

C. To store user login information

D. To encrypt messages for security

Solution

1. Step 1: Understand the role of work queues

   Work queues are designed to share tasks among many workers so that the workload is balanced and tasks are processed efficiently.

2. Step 2: Eliminate unrelated options

   Storing login info, backing up messages, or encrypting messages are not the primary functions of work queues.

3. Final Answer:

   To distribute tasks among multiple workers safely -> Option B

4. Quick Check:

   Work queue = task distribution [OK]

Hint: Work queues share tasks among workers, not store or encrypt data [OK]

Common Mistakes:

• Confusing work queues with message encryption
• Thinking work queues store user data
• Assuming work queues backup messages automatically

2. Which RabbitMQ queue declaration option makes the queue survive server restarts?

easy

A. durable: true

B. autoDelete: true

C. exclusive: true

D. passive: true

Solution

1. Step 1: Identify durability option

   The 'durable: true' option ensures the queue is saved to disk and survives RabbitMQ server restarts.

2. Step 2: Understand other options

   'autoDelete' removes the queue when no consumers remain, 'exclusive' restricts queue to one connection, and 'passive' checks for queue existence without creating it.

3. Final Answer:

   durable: true -> Option A

4. Quick Check:

   durable = survive restart [OK]

Hint: Durable queues survive restarts; autoDelete removes queues [OK]

Common Mistakes:

• Using autoDelete instead of durable for persistence
• Confusing exclusive with durability
• Thinking passive creates durable queues

3. Given this worker code snippet in Python using pika:

channel.basic_consume(queue='task_queue', on_message_callback=callback, auto_ack=False)
channel.start_consuming()

What happens if the worker crashes before sending an acknowledgment?

medium

A. The message is requeued and delivered to another worker

B. The queue deletes all messages

C. The message is acknowledged automatically

D. The message is lost and never processed again

Solution

1. Step 1: Understand auto_ack=False behavior

   With auto_ack set to False, the worker must explicitly acknowledge messages after processing.

2. Step 2: Effect of worker crash before ack

   If the worker crashes before ack, RabbitMQ requeues the message to deliver it to another worker, ensuring no task is lost.

3. Final Answer:

   The message is requeued and delivered to another worker -> Option A

4. Quick Check:

   auto_ack=False + crash = requeue message [OK]

Hint: auto_ack=False means unacked messages get requeued on crash [OK]

Common Mistakes:

• Assuming messages are lost without ack
• Thinking messages auto-acknowledge on delivery
• Believing queue deletes messages on worker failure

4. You wrote this code to declare a durable queue:

channel.queue_declare(queue='task_queue', durable=False)

Why might this cause task loss after RabbitMQ restarts?

medium

A. Because exclusive=True is missing

B. Because the queue name is invalid

C. Because durable=True deletes messages on restart

D. Because durable=False means the queue is deleted on restart

Solution

1. Step 1: Analyze durable flag effect

   Setting durable=False means the queue is temporary and will be deleted when RabbitMQ restarts.

2. Step 2: Consequence on task persistence

   Since the queue is deleted, any unprocessed tasks in it are lost after restart.

3. Final Answer:

   Because durable=False means the queue is deleted on restart -> Option D

4. Quick Check:

   durable=False = queue lost on restart [OK]

Hint: durable=False queues vanish on restart, losing tasks [OK]

Common Mistakes:

• Confusing durable with exclusive
• Thinking queue name causes loss
• Believing durable=True deletes messages

5. You want to ensure tasks are not lost even if RabbitMQ crashes and workers fail. Which combination of settings should you use for the queue and messages?

hard

A. Declare queue with autoDelete=true and publish messages with delivery_mode=1

B. Declare queue with exclusive=true and publish messages with delivery_mode=1

C. Declare queue with durable=true and publish messages with delivery_mode=2

D. Declare queue with durable=false and publish messages with delivery_mode=2

Solution

1. Step 1: Ensure queue durability

   Setting durable=true makes the queue survive RabbitMQ restarts, preventing loss of the queue itself.

2. Step 2: Make messages persistent

   Publishing messages with delivery_mode=2 marks them as persistent, so they are saved to disk and not lost on server crash.

3. Step 3: Understand why other options fail

   autoDelete=true deletes the queue when no consumers remain, exclusive=true restricts queue to one connection, and durable=false means queue is temporary. delivery_mode=1 means messages are transient and can be lost.

4. Final Answer:

   Declare queue with durable=true and publish messages with delivery_mode=2 -> Option C

5. Quick Check:

   durable queue + persistent messages = no loss [OK]

Hint: Durable queue + delivery_mode=2 messages prevent loss [OK]

Common Mistakes:

• Using autoDelete instead of durable for persistence
• Publishing messages without delivery_mode=2
• Confusing exclusive with durability