Parallel execution in CI in PyTest - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Understanding pytest-xdist parallel test execution output

Given the pytest command below executed with pytest-xdist for parallel testing:

pytest -n 3 test_sample.py

What is the expected output behavior regarding test distribution?

PyTest

import time
import pytest

def test_one():
    time.sleep(1)
    assert True

def test_two():
    time.sleep(1)
    assert True

def test_three():
    time.sleep(1)
    assert True

A Tests run sequentially but pytest-xdist reports parallel execution without speedup.

B All three tests run sequentially on the same worker, taking about 3 seconds total.

C Tests run in parallel but only two workers are used, taking about 2 seconds total.

D Tests run in parallel across 3 workers, completing in about 1 second total.

💡 Hint

pytest-xdist distributes tests across workers to run them simultaneously.

Using '-n 3' with pytest-xdist runs tests on 3 parallel workers. Each test sleeps 1 second, so total time is about 1 second, not 3.

❓ assertion

intermediate

Choosing the correct assertion for parallel test logs

In a CI pipeline running tests in parallel, which assertion best verifies that all test logs contain the string 'PASSED' exactly once per test?

PyTest

logs = ['test_one PASSED', 'test_two PASSED', 'test_three PASSED']

A assert logs == ['PASSED', 'PASSED', 'PASSED']

B assert all('PASSED' in log for log in logs)

C assert logs.count('PASSED') == 3

D assert any('PASSED' in log for log in logs)

💡 Hint

Check each log individually for the substring.

Option B checks that every log string contains 'PASSED'. Option B is invalid because logs is a list of strings, not a list of 'PASSED'. Option B expects exact strings, which is incorrect. Option B only checks if any log contains 'PASSED', not all.

🔧 Debug

advanced

Debugging flaky tests in parallel CI runs

A test suite runs fine locally but intermittently fails in CI when run with pytest-xdist parallel execution. Which is the most likely cause?

A Tests share global state or resources causing race conditions.

B pytest-xdist does not support parallel execution on CI servers.

C The test code has syntax errors that only appear in CI.

D The CI server has insufficient CPU cores to run tests.

💡 Hint

Parallel tests must avoid shared mutable state.

Flaky failures in parallel runs often come from tests sharing global variables or files, causing race conditions. pytest-xdist supports parallel runs on CI. Syntax errors cause consistent failures. CPU cores affect speed but not correctness.

❓ framework

advanced

Configuring pytest for parallel execution in CI

Which pytest configuration snippet correctly enables parallel test execution with 4 workers in a CI environment?

A

[pytest]
addopts = -n 4 --dist=loadfile

B

[pytest]
addopts = --workers=4 --dist=loadfile

C

[pytest]
addopts = -n 4 --dist=loadscope

D

[pytest]
addopts = --parallel=4 --dist=loadscope

💡 Hint

The '-n' option sets the number of workers.

Option A uses '-n 4' to specify 4 workers and '--dist=loadfile' to distribute tests by file, which is valid. Option A uses invalid '--dist=loadscope'. Option A uses invalid '--workers'. Option A uses invalid '--parallel'.

🧠 Conceptual

expert

Impact of parallel test execution on test isolation in CI

Why is test isolation especially critical when running tests in parallel in a CI pipeline?

A Because isolated tests require more CPU resources in parallel.

B Because parallel tests always run slower, isolation improves speed.

C Because parallel tests may run simultaneously, shared state can cause unpredictable failures.

D Because CI pipelines do not support parallel execution without isolation.

💡 Hint

Think about what happens if tests share data at the same time.

Parallel tests run at the same time, so if they share state or resources, they can interfere and cause random failures. Isolation prevents this by keeping tests independent.

Practice

1. What is the main benefit of using parallel execution in pytest within a CI environment?

easy

A. It disables flaky tests to improve stability.

B. It automatically fixes failing tests during execution.

C. It generates detailed test coverage reports.

D. It runs multiple tests at the same time to reduce total test time.

Solution

1. Step 1: Understand parallel execution purpose

   Parallel execution means running tests simultaneously instead of one by one.

2. Step 2: Identify benefit in CI context

   Running tests at the same time reduces the total time needed to finish all tests in CI.

3. Final Answer:

   It runs multiple tests at the same time to reduce total test time. -> Option D

4. Quick Check:

   Parallel execution = faster test runs [OK]

Hint: Parallel means multiple tests run together, saving time [OK]

Common Mistakes:

• Confusing parallel execution with automatic bug fixing
• Thinking it generates reports automatically
• Assuming it disables tests instead of running them

2. Which command correctly enables parallel test execution using pytest-xdist with 4 workers?

easy

A. pytest -n 4

B. pytest --workers=4

C. pytest --parallel=4

D. pytest -p xdist 4

Solution

1. Step 1: Recall pytest-xdist syntax

   The pytest-xdist plugin uses the option -n followed by the number of workers.

2. Step 2: Match correct command

   The correct command to run tests in parallel with 4 workers is pytest -n 4.

3. Final Answer:

   pytest -n 4 -> Option A

4. Quick Check:

   Use -n to set worker count [OK]

Hint: Remember: -n sets number of parallel workers [OK]

Common Mistakes:

• Using --workers instead of -n
• Adding number without -n option
• Misplacing plugin name in command

3. Given this pytest command in CI: pytest -n 3 tests/, what is the expected behavior?

medium

A. Tests in the 'tests/' folder run sequentially on one worker.

B. Tests in the 'tests/' folder run in parallel on 3 workers.

C. Only 3 tests will run from the 'tests/' folder.

D. Tests will run with 3 retries on failure.

Solution

1. Step 1: Analyze the command options

   The -n 3 option tells pytest-xdist to use 3 parallel workers.

2. Step 2: Understand test execution effect

   All tests in the 'tests/' folder will be distributed and run simultaneously on 3 workers.

3. Final Answer:

   Tests in the 'tests/' folder run in parallel on 3 workers. -> Option B

4. Quick Check:

   -n 3 means 3 parallel workers [OK]

Hint: -n 3 means run tests on 3 parallel workers [OK]

Common Mistakes:

• Thinking only 3 tests run total
• Assuming tests run sequentially
• Confusing retries with parallelism

4. You added pytest -n 4 in your CI but tests still run sequentially. What is the most likely cause?

medium

A. The tests folder is empty, so no tests run.

B. You need to add --parallel option instead of -n.

C. pytest-xdist plugin is not installed in the CI environment.

D. You must specify the number of retries for parallel to work.

Solution

1. Step 1: Check plugin requirement for parallelism

   pytest-xdist plugin must be installed to enable -n parallel execution.

2. Step 2: Identify cause of sequential runs

   If plugin is missing, pytest ignores -n and runs tests sequentially.

3. Final Answer:

   pytest-xdist plugin is not installed in the CI environment. -> Option C

4. Quick Check:

   Missing plugin causes no parallelism [OK]

Hint: Parallel needs pytest-xdist installed to work [OK]

Common Mistakes:

• Using wrong option like --parallel
• Assuming empty folder causes sequential runs
• Confusing retries with parallel execution

5. In a CI pipeline, you want to run tests in parallel but limit each worker to use only one CPU core to avoid overload. Which pytest-xdist option helps achieve this?

hard

A. Use pytest -n auto --dist loadscope to auto assign workers with load balancing.

B. Use pytest -n 4 --max-worker-threads=1 to limit threads per worker.

C. Use pytest -n 4 --boxed to isolate each test in a subprocess.

D. Use pytest -n 4 --max-worker-memory=1G to limit memory per worker.

Solution

1. Step 1: Understand CPU core limitation in pytest-xdist

   pytest-xdist can auto detect CPU cores and assign workers accordingly using -n auto.

2. Step 2: Use load balancing to distribute tests efficiently

   The --dist loadscope option balances tests to avoid overloading any worker.

3. Final Answer:

   Use pytest -n auto --dist loadscope to auto assign workers with load balancing. -> Option A

4. Quick Check:

   -n auto with loadscope balances CPU load [OK]

Hint: -n auto with --dist loadscope balances CPU load per worker [OK]

Common Mistakes:

• Using non-existent options like --max-worker-threads
• Confusing --boxed with CPU core limits
• Trying to limit memory instead of CPU