How to Use pytest-xdist for Parallel Test Execution

Use pytest-xdist by installing it and running tests with the -n option followed by the number of parallel workers, like pytest -n 4. This runs tests concurrently across multiple CPU cores, speeding up test execution.

Syntax

The basic syntax to run tests in parallel with pytest-xdist is:

• pytest -n NUM: Runs tests using NUM parallel workers.
• NUM can be any positive integer or auto to use all available CPU cores.

Example: pytest -n 4 runs tests on 4 parallel workers.

bash

pytest -n 4

Example

This example shows a simple test file with multiple tests and how to run them in parallel using pytest-xdist.

python

import time
import pytest

def test_one():
    time.sleep(1)
    assert True

def test_two():
    time.sleep(1)
    assert True

def test_three():
    time.sleep(1)
    assert True

def test_four():
    time.sleep(1)
    assert True

# Run this with: pytest -n 4 test_parallel.py

Output

============================= test session starts ==============================
platform linux -- Python 3.x.x, pytest-7.x.x, pluggy-1.x.x
rootdir: /path/to/tests
plugins: xdist-3.x.x
gw0 [4] / gw1 [4] / gw2 [4] / gw3 [4]

collected 4 items

test_parallel.py::test_one PASSED                                         [ 25%]
test_parallel.py::test_two PASSED                                         [ 50%]
test_parallel.py::test_three PASSED                                       [ 75%]
test_parallel.py::test_four PASSED                                        [100%]

============================== 4 passed in 1.05s ===============================

Common Pitfalls

• Not installing pytest-xdist: You must install it with pip install pytest-xdist before using -n.
• Using shared resources: Tests that modify the same file or database can fail when run in parallel.
• Tests depending on order: Parallel runs do not guarantee test order, so tests must be independent.

bash

## Wrong: Running without pytest-xdist installed
# Command: pytest -n 4
# Error: pytest: error: unrecognized arguments: -n 4

## Right: Install first
# pip install pytest-xdist
# pytest -n 4

Quick Reference

Command	Description
pytest -n 4	Run tests with 4 parallel workers
pytest -n auto	Run tests using all CPU cores automatically
pip install pytest-xdist	Install the pytest-xdist plugin
pytest --dist=loadscope -n 4	Distribute tests by scope to reduce conflicts

Key Takeaways

Install pytest-xdist with pip before using parallel options.

Use pytest -n NUM to run tests in parallel with NUM workers.

Ensure tests are independent and do not share mutable state.

Use -n auto to utilize all CPU cores automatically.

Parallel testing speeds up test suites but requires careful test design.