Parallel execution in CI in PyTest

Introduction

Running tests in parallel helps finish testing faster. It saves time by doing many tests at once instead of one by one.

You have many tests and want to get results quickly.

Your CI system takes too long to finish tests.

You want to use multiple CPU cores to speed up testing.

You want to find problems that happen when tests run at the same time.

You want to improve feedback speed for developers after code changes.

Syntax

PyTest

pytest -n <number_of_workers>

Use the -n option with pytest to specify how many tests run at the same time.

You need to install the pytest-xdist plugin to enable parallel execution.

Examples

This runs tests using 4 workers in parallel.

PyTest

pytest -n 4

This runs tests using as many workers as CPU cores available.

PyTest

pytest -n auto

Sample Program

This example has 4 tests that each wait 1 second. Running with -n 4 runs them all at once, so total time is about 1 second.

PyTest

import time
import pytest

def test_sleep_1():
    time.sleep(1)
    assert True

def test_sleep_2():
    time.sleep(1)
    assert True

def test_sleep_3():
    time.sleep(1)
    assert True

def test_sleep_4():
    time.sleep(1)
    assert True

# Run command:
# pytest -n 4

# Expected behavior:
# All 4 tests run at the same time, total time about 1 second instead of 4 seconds.

Important Notes

Make sure tests do not share or change the same data when running in parallel to avoid conflicts.

Use fixtures with scope='function' to isolate test data.

Check your CI system supports parallel jobs or configure it to allow multiple workers.

Summary

Parallel execution runs tests at the same time to save time.

Use pytest -n with pytest-xdist plugin to enable it.

Good for speeding up CI and handling many tests efficiently.