pytest-xdist for parallel execution - Build an Automation Script

Run multiple pytest tests in parallel using pytest-xdist

Preconditions (2)

Step 1: Open a terminal or command prompt

Step 2: Navigate to the directory containing test_sample.py

Step 3: Run the command: pytest -n 3

Step 4: Observe the test execution running in parallel across 3 workers

Step 5: Verify that all tests pass successfully

✅ Expected Result: All tests in test_sample.py run in parallel using 3 workers and all tests pass without errors

Automation Requirements - pytest

Assertions Needed:

Verify that all tests pass

Verify that tests run in parallel (e.g., by checking timestamps or pytest-xdist output)

Best Practices:

Use pytest markers or fixtures to ensure tests are independent

Avoid shared state between tests to prevent flaky results

Use pytest's built-in assertion rewriting for clear error messages

Automated Solution

PyTest

import subprocess
import re

def test_pytest_xdist_parallel_execution():
    # Run pytest with 3 parallel workers
    result = subprocess.run(['pytest', '-n', '3', 'test_sample.py', '--maxfail=1', '--disable-warnings', '-q'], capture_output=True, text=True)
    output = result.stdout
    # Check that pytest exited with code 0 (all tests passed)
    assert result.returncode == 0, f"Tests failed with output:\n{output}"
    # Check pytest-xdist output for parallel workers
    # The output usually contains lines like 'gw0', 'gw1', 'gw2' indicating workers
    workers_found = re.findall(r'gw\d+', output)
    assert len(set(workers_found)) >= 3, "Less than 3 parallel workers detected in output"

This test runs the pytest command with the -n 3 option to execute tests in 3 parallel workers using pytest-xdist.

We use subprocess.run to execute the command and capture the output.

First, we assert that the return code is 0, meaning all tests passed.

Then, we check the output for worker identifiers like gw0, gw1, and gw2 which pytest-xdist prints to show parallel execution.

This confirms that tests ran in parallel.

Using --maxfail=1 and --disable-warnings keeps output clean and stops early on failure.

Common Mistakes - 3 Pitfalls

Running tests with pytest-xdist without ensuring tests are independent

{'mistake': 'Not capturing or checking the pytest output for parallel execution confirmation', 'why_bad': 'You might think tests ran in parallel but they actually ran sequentially', 'correct_approach': "Parse pytest output for worker identifiers like 'gw0', 'gw1' to confirm parallelism"}

{'mistake': "Using hardcoded sleep or wait instead of relying on pytest-xdist's parallelism", 'why_bad': 'Sleep does not guarantee parallel execution and slows tests unnecessarily', 'correct_approach': "Use pytest-xdist's -n option to run tests in parallel properly"}

Bonus Challenge

Now add a test that runs the same test file with different numbers of workers (1, 2, 4) and verifies all pass

Show Hint