Bird
Raised Fist0
PyTesttesting~15 mins

Worker distribution strategies in PyTest - Build an Automation Script

Choose your learning style10 modes available

Start learning this pattern below

Jump into concepts and practice - no test required

or
Recommended
Test this pattern10 questions across easy, medium, and hard to know if this pattern is strong
Verify pytest worker distribution with multiple tests
Preconditions (2)
Step 1: Run pytest with the -n 2 option to use 2 workers
Step 2: Observe that tests are distributed between the two workers
Step 3: Run pytest with the -n 4 option to use 4 workers
Step 4: Observe that tests are distributed among the four workers
Step 5: Run pytest with the --dist=loadscope option and 2 workers
Step 6: Observe that tests grouped by scope are distributed to workers
✅ Expected Result: Tests are split and run in parallel across the specified number of workers according to the distribution strategy, with no test failures due to worker distribution.
Automation Requirements - pytest
Assertions Needed:
Verify all tests pass when run with multiple workers
Verify tests are executed in parallel by checking timestamps or logs
Verify no test is skipped or duplicated
Best Practices:
Use pytest-xdist plugin for parallel execution
Use fixtures with appropriate scope to avoid conflicts
Avoid shared state between tests to prevent race conditions
Use explicit markers or grouping if using loadscope distribution
Automated Solution
PyTest
import pytest
import time
import threading

execution_log = []

@pytest.fixture(scope='module')
def resource():
    return "shared_resource"

@pytest.mark.parametrize('i', range(4))
def test_parallel_execution(i, resource):
    start = time.time()
    thread_name = threading.current_thread().name
    execution_log.append((i, thread_name, start))
    time.sleep(0.5)  # Simulate work
    end = time.time()
    execution_log.append((i, thread_name, end))
    assert resource == "shared_resource"


def test_all_tests_ran():
    # Check that all 4 tests ran
    test_ids = set(entry[0] for entry in execution_log)
    assert test_ids == {0,1,2,3}

# To run this test suite with parallel workers, use:
# pytest -n 2 --dist=loadscope
# or
# pytest -n 4

# The assertions verify all tests ran and used the shared fixture correctly.

The test suite defines 4 parameterized tests to simulate independent tests.

Each test logs its start and end time with the thread name to show parallel execution.

The shared fixture with module scope ensures no conflicts.

The final test verifies all tests ran by checking the log.

Running with pytest -n 2 or pytest -n 4 uses multiple workers to distribute tests.

The --dist=loadscope option groups tests by scope for distribution.

This setup follows best practices by avoiding shared state conflicts and verifying parallel execution.

Common Mistakes - 4 Pitfalls
Not installing or enabling pytest-xdist plugin
Using shared mutable state without proper isolation
Hardcoding sleep times without synchronization
Not verifying that all tests ran and none were skipped or duplicated
Bonus Challenge

Now add data-driven testing with 3 different inputs for each test and verify parallel execution with workers

Show Hint

Practice

(1/5)
1. What does the --dist=loadscope option do in pytest-xdist worker distribution?
easy
A. It distributes tests randomly to all workers.
B. It runs all tests sequentially on a single worker.
C. It groups tests by their scope and distributes them to workers.
D. It groups tests by file size before distribution.

Solution

  1. Step 1: Understand the meaning of loadscope

    The loadscope mode groups tests by their scope, such as class or module, so related tests run together.
  2. Step 2: Compare with other distribution modes

    Unlike random or file-based grouping, loadscope keeps related tests together for better caching and setup reuse.
  3. Final Answer:

    It groups tests by their scope and distributes them to workers. -> Option C
  4. Quick Check:

    loadscope = group by scope [OK]
Hint: Loadscope groups tests by scope like class or module [OK]
Common Mistakes:
  • Confusing loadscope with random distribution
  • Thinking loadscope groups by file size
  • Assuming loadscope runs tests sequentially
2. Which of the following is the correct pytest command to run tests with 4 workers using file-based distribution?
easy
A. pytest -n 4 --dist=loadfile
B. pytest --dist=loadfile -n four
C. pytest -n=4 --dist=loadscope
D. pytest -n 4 --dist=loadgroup

Solution

  1. Step 1: Identify correct syntax for number of workers

    The correct syntax is -n 4 to specify 4 workers; spelling out 'four' is invalid.
  2. Step 2: Match distribution mode to file-based

    The file-based distribution mode is loadfile, so --dist=loadfile is correct.
  3. Final Answer:

    pytest -n 4 --dist=loadfile -> Option A
  4. Quick Check:

    -n 4 and --dist=loadfile correct syntax [OK]
Hint: Use -n number and --dist=loadfile for file grouping [OK]
Common Mistakes:
  • Using spelled-out numbers like 'four'
  • Mixing distribution modes incorrectly
  • Using equals sign with -n option
3. Given this pytest command: pytest -n 3 --dist=loadfile, and three test files test_a.py, test_b.py, test_c.py, how will tests be distributed?
medium
A. Tests run sequentially on a single worker.
B. All workers run tests from all files randomly.
C. Tests are grouped by class across files.
D. Each worker runs tests from one file exclusively.

Solution

  1. Step 1: Understand loadfile distribution

    Loadfile mode assigns tests grouped by file to different workers, so each worker gets whole files.
  2. Step 2: Match number of workers to files

    With 3 workers and 3 files, each worker will get one file's tests exclusively.
  3. Final Answer:

    Each worker runs tests from one file exclusively. -> Option D
  4. Quick Check:

    loadfile = group by file [OK]
Hint: Loadfile means one file per worker [OK]
Common Mistakes:
  • Thinking tests are split randomly
  • Confusing loadfile with loadscope
  • Assuming tests run sequentially
4. You run pytest -n 2 --dist=loadscope but notice tests from the same class run on different workers. What is the likely cause?
medium
A. Tests are not properly grouped because the class scope is not detected.
B. The -n option must be set to 1 for loadscope.
C. The --dist option is ignored when using multiple workers.
D. Tests are always distributed randomly regardless of options.

Solution

  1. Step 1: Understand loadscope grouping behavior

    Loadscope groups tests by scope like class or module, so tests in the same class should run together.
  2. Step 2: Identify why grouping fails

    If tests from the same class run on different workers, pytest likely failed to detect the class scope properly, causing wrong grouping.
  3. Final Answer:

    Tests are not properly grouped because the class scope is not detected. -> Option A
  4. Quick Check:

    Undetected scope breaks loadscope grouping [OK]
Hint: Undetected scope causes loadscope to fail grouping [OK]
Common Mistakes:
  • Thinking -n must be 1 for loadscope
  • Believing --dist is ignored with multiple workers
  • Assuming distribution is always random
5. You want to run tests in custom groups using pytest-xdist. Which command and option combination allows you to define and use custom test groups for worker distribution?
hard
A. pytest -n 3 --dist=loadgroup --tx group1 --tx group2 --tx group3
B. pytest -n 3 --dist=loadgroup
C. pytest -n 3 --dist=loadfile --group=custom
D. pytest -n 3 --dist=loadscope --group=custom

Solution

  1. Step 1: Identify the distribution mode for custom groups

    The loadgroup mode is designed for custom grouping of tests for distribution.
  2. Step 2: Understand correct command usage

    Using --dist=loadgroup with -n 3 enables pytest-xdist to distribute tests based on user-defined groups configured elsewhere (e.g., in pytest hooks).
  3. Final Answer:

    pytest -n 3 --dist=loadgroup -> Option B
  4. Quick Check:

    loadgroup enables custom test groups [OK]
Hint: Use --dist=loadgroup to enable custom test groups [OK]
Common Mistakes:
  • Adding invalid --group option
  • Using --tx incorrectly for grouping
  • Confusing loadgroup with loadfile or loadscope