Worker distribution strategies in PyTest - Framework Patterns

Framework Mode - Worker distribution strategies

Folder Structure

tests/
├── test_login.py
├── test_checkout.py
├── test_search.py
conftest.py
pytest.ini
requirements.txt

# Optional for parallel execution
pytest_workers/
├── worker_1/
├── worker_2/
└── worker_n/

Test Framework Layers

• Test Cases: Located in tests/ folder, contain test functions using pytest syntax.
• Fixtures & Hooks: Defined in conftest.py for setup/teardown and sharing resources.
• Configuration: pytest.ini or pyproject.toml to configure pytest options including markers and parallel workers.
• Worker Distribution: Managed by pytest-xdist plugin, which distributes tests across multiple CPU cores or machines.
• Utilities: Helper functions or modules imported by tests or fixtures.

Configuration Patterns

• pytest.ini example to enable parallel workers:

  [pytest]
  addopts = -n auto  # Automatically use all CPU cores
  markers =
      smoke: smoke tests
      regression: regression tests
  

• Environment Handling: Use environment variables or pytest command line options to select environments (dev, staging, prod).
• Credentials: Store securely outside code, load in fixtures using environment variables or config files.
• Selective Distribution: Use pytest markers to group tests and distribute workers accordingly.

Test Reporting and CI/CD Integration

• Use pytest built-in reports with --junitxml=report.xml for CI systems.
• Integrate with CI tools like GitHub Actions, Jenkins, GitLab CI to run tests in parallel using pytest-xdist.
• Generate HTML reports with plugins like pytest-html for easy visualization.
• Configure CI to split tests across workers to reduce total test time.

Best Practices

1. Use pytest-xdist plugin for easy parallel test execution and worker distribution.
2. Keep tests independent so they can run in any order or on any worker without side effects.
3. Use markers to group tests logically and control which tests run on which workers.
4. Manage shared resources carefully to avoid conflicts when tests run in parallel.
5. Configure workers dynamically based on available CPU cores or CI environment variables.

Self Check

Where in this framework structure would you add a new fixture that sets up a database connection shared by tests running on different workers?

Key Result

Use pytest-xdist to distribute tests across multiple workers for faster parallel execution while keeping tests independent and well-organized.

Practice

1. What does the --dist=loadscope option do in pytest-xdist worker distribution?

easy

A. It distributes tests randomly to all workers.

B. It runs all tests sequentially on a single worker.

C. It groups tests by their scope and distributes them to workers.

D. It groups tests by file size before distribution.

Solution

1. Step 1: Understand the meaning of loadscope

   The loadscope mode groups tests by their scope, such as class or module, so related tests run together.

2. Step 2: Compare with other distribution modes

   Unlike random or file-based grouping, loadscope keeps related tests together for better caching and setup reuse.

3. Final Answer:

   It groups tests by their scope and distributes them to workers. -> Option C

4. Quick Check:

   loadscope = group by scope [OK]

Hint: Loadscope groups tests by scope like class or module [OK]

Common Mistakes:

• Confusing loadscope with random distribution
• Thinking loadscope groups by file size
• Assuming loadscope runs tests sequentially

2. Which of the following is the correct pytest command to run tests with 4 workers using file-based distribution?

easy

A. pytest -n 4 --dist=loadfile

B. pytest --dist=loadfile -n four

C. pytest -n=4 --dist=loadscope

D. pytest -n 4 --dist=loadgroup

Solution

1. Step 1: Identify correct syntax for number of workers

   The correct syntax is -n 4 to specify 4 workers; spelling out 'four' is invalid.

2. Step 2: Match distribution mode to file-based

   The file-based distribution mode is loadfile, so --dist=loadfile is correct.

3. Final Answer:

   pytest -n 4 --dist=loadfile -> Option A

4. Quick Check:

   -n 4 and --dist=loadfile correct syntax [OK]

Hint: Use -n number and --dist=loadfile for file grouping [OK]

Common Mistakes:

• Using spelled-out numbers like 'four'
• Mixing distribution modes incorrectly
• Using equals sign with -n option

3. Given this pytest command: pytest -n 3 --dist=loadfile, and three test files test_a.py, test_b.py, test_c.py, how will tests be distributed?

medium

A. Tests run sequentially on a single worker.

B. All workers run tests from all files randomly.

C. Tests are grouped by class across files.

D. Each worker runs tests from one file exclusively.

Solution

1. Step 1: Understand loadfile distribution

   Loadfile mode assigns tests grouped by file to different workers, so each worker gets whole files.

2. Step 2: Match number of workers to files

   With 3 workers and 3 files, each worker will get one file's tests exclusively.

3. Final Answer:

   Each worker runs tests from one file exclusively. -> Option D

4. Quick Check:

   loadfile = group by file [OK]

Hint: Loadfile means one file per worker [OK]

Common Mistakes:

• Thinking tests are split randomly
• Confusing loadfile with loadscope
• Assuming tests run sequentially

4. You run pytest -n 2 --dist=loadscope but notice tests from the same class run on different workers. What is the likely cause?

medium

A. Tests are not properly grouped because the class scope is not detected.

B. The -n option must be set to 1 for loadscope.

C. The --dist option is ignored when using multiple workers.

D. Tests are always distributed randomly regardless of options.

Solution

1. Step 1: Understand loadscope grouping behavior

   Loadscope groups tests by scope like class or module, so tests in the same class should run together.

2. Step 2: Identify why grouping fails

   If tests from the same class run on different workers, pytest likely failed to detect the class scope properly, causing wrong grouping.

3. Final Answer:

   Tests are not properly grouped because the class scope is not detected. -> Option A

4. Quick Check:

   Undetected scope breaks loadscope grouping [OK]

Hint: Undetected scope causes loadscope to fail grouping [OK]

Common Mistakes:

• Thinking -n must be 1 for loadscope
• Believing --dist is ignored with multiple workers
• Assuming distribution is always random

5. You want to run tests in custom groups using pytest-xdist. Which command and option combination allows you to define and use custom test groups for worker distribution?

hard

A. pytest -n 3 --dist=loadgroup --tx group1 --tx group2 --tx group3

B. pytest -n 3 --dist=loadgroup

C. pytest -n 3 --dist=loadfile --group=custom

D. pytest -n 3 --dist=loadscope --group=custom

Solution

1. Step 1: Identify the distribution mode for custom groups

   The loadgroup mode is designed for custom grouping of tests for distribution.

2. Step 2: Understand correct command usage

   Using --dist=loadgroup with -n 3 enables pytest-xdist to distribute tests based on user-defined groups configured elsewhere (e.g., in pytest hooks).

3. Final Answer:

   pytest -n 3 --dist=loadgroup -> Option B

4. Quick Check:

   loadgroup enables custom test groups [OK]

Hint: Use --dist=loadgroup to enable custom test groups [OK]

Common Mistakes:

• Adding invalid --group option
• Using --tx incorrectly for grouping
• Confusing loadgroup with loadfile or loadscope