What is Multiprocessing in Python: Explanation and Example

In Python, multiprocessing is a module that allows running multiple processes at the same time to perform tasks in parallel. It helps use multiple CPU cores to speed up programs by running separate tasks independently.

How It Works

Multiprocessing in Python works by creating separate processes that run independently from each other. Imagine you have several workers in a kitchen, each cooking a different dish at the same time instead of one cook doing everything one after another. This way, the work gets done faster.

Each process has its own memory space, so they don't share variables directly. They communicate by sending messages or using special shared objects. This avoids problems that happen when multiple tasks try to change the same data at once.

Example

This example shows how to use the multiprocessing module to run two tasks at the same time. Each task prints a message and waits for a moment.

python

import multiprocessing
import time

def worker(name):
    print(f"Worker {name} starting")
    time.sleep(2)
    print(f"Worker {name} done")

if __name__ == "__main__":
    p1 = multiprocessing.Process(target=worker, args=("A",))
    p2 = multiprocessing.Process(target=worker, args=("B",))

    p1.start()
    p2.start()

    p1.join()
    p2.join()

    print("Both workers finished")

Output

Worker A starting
Worker B starting
Worker A done
Worker B done
Both workers finished

When to Use

Use multiprocessing when you want to speed up programs that do heavy work like calculations, data processing, or tasks that can run separately without waiting for each other. It is especially helpful on computers with multiple CPU cores.

For example, if you have a program that processes many images or runs simulations, multiprocessing can run several parts at once to finish faster. It is less useful for tasks that mostly wait for input/output, like reading files or web requests.

Key Points

• Multiprocessing runs multiple processes in parallel using separate memory.
• It helps use multiple CPU cores to speed up CPU-heavy tasks.
• Processes do not share memory directly, avoiding conflicts.
• Communication between processes requires special methods.
• Best for tasks that can run independently and need more CPU power.

Key Takeaways

Multiprocessing runs multiple processes at the same time to use multiple CPU cores.

Each process has its own memory, so they don't interfere with each other.

Use multiprocessing to speed up CPU-heavy tasks that can run independently.

Communication between processes needs special tools like queues or pipes.

It is less effective for tasks that mostly wait for input/output operations.