How to Use Queue for Thread Communication in Python

Use Python's queue.Queue to safely share data between threads. One thread puts data into the queue with put(), and another thread retrieves it with get(), ensuring thread-safe communication.

Syntax

The queue.Queue class provides a thread-safe FIFO queue. Key methods include:

• put(item): Adds an item to the queue.
• get(): Removes and returns an item from the queue, blocking if empty.
• task_done(): Signals that a formerly enqueued task is complete.
• join(): Blocks until all items have been processed.

python

import queue

q = queue.Queue()

# Add item
q.put('data')

# Remove item
item = q.get()

# Signal task done
q.task_done()

# Wait for all tasks
q.join()

Example

This example shows two threads communicating via a queue: one thread puts numbers into the queue, and the other thread reads and prints them.

python

import threading
import queue
import time

def producer(q):
    for i in range(5):
        print(f'Producing {i}')
        q.put(i)
        time.sleep(0.5)
    q.put(None)  # Signal consumer to stop

def consumer(q):
    while True:
        item = q.get()
        if item is None:
            q.task_done()
            break
        print(f'Consuming {item}')
        q.task_done()

q = queue.Queue()

producer_thread = threading.Thread(target=producer, args=(q,))
consumer_thread = threading.Thread(target=consumer, args=(q,))

producer_thread.start()
consumer_thread.start()

producer_thread.join()
consumer_thread.join()

Output

Producing 0
Consuming 0
Producing 1
Consuming 1
Producing 2
Consuming 2
Producing 3
Consuming 3
Producing 4
Consuming 4

Common Pitfalls

Common mistakes when using queues for thread communication include:

• Not using None or a sentinel value to signal the consumer thread to stop, causing it to block forever.
• Calling get() without handling empty queues or blocking properly.
• Not calling task_done() after processing an item, which can cause join() to block indefinitely.

python

import queue
import threading

q = queue.Queue()

# Wrong: consumer blocks forever without sentinel
# def consumer():
#     while True:
#         item = q.get()
#         print(item)

# Right: use sentinel None to stop

def consumer():
    while True:
        item = q.get()
        if item is None:
            q.task_done()
            break
        print(item)
        q.task_done()

Quick Reference

Tips for using queue.Queue in thread communication:

• Use put() to send data safely between threads.
• Use get() to receive data, it blocks if empty.
• Use a sentinel value like None to signal consumer thread to stop.
• Always call task_done() after processing an item if you use join().

Key Takeaways

Use queue.Queue to safely share data between threads without conflicts.

Producer threads use put() to add items; consumer threads use get() to retrieve them.

Use a sentinel value like None to signal consumers to stop processing.

Call task_done() after processing each item to avoid blocking join().

Queue methods handle locking internally, so no extra locks are needed.