How to Create Thread Pool in Python: Simple Guide

In Python, you can create a thread pool using concurrent.futures.ThreadPoolExecutor. This lets you run multiple tasks in parallel threads easily by submitting functions to the pool.

Syntax

The basic syntax to create a thread pool uses ThreadPoolExecutor from the concurrent.futures module. You specify the maximum number of worker threads with max_workers. Then you submit tasks (functions) to run concurrently.

• ThreadPoolExecutor(max_workers=n): Creates a pool with n threads.
• submit(fn, *args): Adds a function fn with arguments args to the pool.
• shutdown(): Cleans up the pool after tasks finish.

python

from concurrent.futures import ThreadPoolExecutor

def task(arg):
    # Your task code here
    return arg * 2

with ThreadPoolExecutor(max_workers=3) as executor:
    future = executor.submit(task, 5)
    result = future.result()
    print(result)

Output

10

Example

This example shows how to create a thread pool with 3 threads and run multiple tasks that print messages with delays. It demonstrates how tasks run in parallel and how to get their results.

python

import time
from concurrent.futures import ThreadPoolExecutor

def print_number(number):
    time.sleep(1)
    print(f"Number: {number}")
    return number * 2

with ThreadPoolExecutor(max_workers=3) as executor:
    futures = [executor.submit(print_number, i) for i in range(5)]
    results = [future.result() for future in futures]

print("Results:", results)

Output

Number: 0
Number: 1
Number: 2
Number: 3
Number: 4
Results: [0, 2, 4, 6, 8]

Common Pitfalls

Common mistakes when using thread pools include:

• Not using with or calling shutdown(), which can leave threads running.
• Submitting blocking or CPU-heavy tasks that don't benefit from threads.
• Ignoring exceptions inside tasks, which can cause silent failures.
• Assuming tasks run in order; threads run asynchronously.

Always handle exceptions inside your task functions and use with to manage the pool.

python

from concurrent.futures import ThreadPoolExecutor

def faulty_task(x):
    if x == 2:
        raise ValueError("Error on 2")
    return x

with ThreadPoolExecutor(max_workers=2) as executor:
    futures = [executor.submit(faulty_task, i) for i in range(4)]
    for future in futures:
        try:
            print(future.result())
        except Exception as e:
            print(f"Caught exception: {e}")

Output

0
1
Caught exception: Error on 2
3

Quick Reference

Here is a quick summary of key methods and parameters for ThreadPoolExecutor:

Method/Parameter	Description
ThreadPoolExecutor(max_workers)	Creates a thread pool with specified number of threads
submit(fn, *args)	Schedules a function to run in the pool
map(fn, iterable)	Runs a function on each item in iterable concurrently
shutdown(wait=True)	Stops the pool and waits for tasks to finish
future.result()	Gets the result of a submitted task, waits if needed

Key Takeaways

Use concurrent.futures.ThreadPoolExecutor to create and manage thread pools easily.

Always use the 'with' statement to ensure proper cleanup of threads.

Submit tasks with submit() and get results with future.result().

Handle exceptions inside tasks to avoid silent failures.

Thread pools are best for I/O-bound or light CPU tasks, not heavy CPU work.