Parallel execution in CI in PyTest - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is parallel execution in Continuous Integration (CI)?

Parallel execution in CI means running multiple tests at the same time to save time and get faster feedback on code changes.

beginner

How does pytest support parallel test execution?

Pytest supports parallel test execution using the plugin called pytest-xdist, which allows tests to run on multiple CPU cores simultaneously.

beginner

What is a common command to run pytest tests in parallel using 4 CPUs?

The command is pytest -n 4, where -n 4 tells pytest to use 4 CPU cores for running tests in parallel.

beginner

Why is parallel execution useful in CI pipelines?

It reduces the total test run time, helping developers get quick feedback and merge code faster, improving productivity and code quality.

intermediate

Name one challenge when running tests in parallel in CI.

Tests might interfere with each other if they share resources like files or databases, causing flaky or failing tests.

Which pytest plugin enables parallel test execution?

A pytest-cov

B pytest-mock

C pytest-html

D pytest-xdist

The pytest-xdist plugin allows tests to run in parallel across multiple CPUs.

What does the command pytest -n 3 do?

A Runs tests sequentially 3 times

B Runs tests on 3 CPU cores in parallel

C Runs only 3 tests

D Runs tests with 3 retries

The -n 3 option tells pytest to run tests in parallel using 3 CPU cores.

Why might tests fail when run in parallel in CI?

A Because tests share resources causing conflicts

B Because tests run too slowly

C Because pytest does not support parallelism

D Because CI servers do not allow parallel jobs

Tests sharing resources like files or databases can interfere with each other when run in parallel.

What is a key benefit of parallel test execution in CI?

A Slower test runs

B More manual testing

C Faster feedback on code changes

D Less test coverage

Parallel execution speeds up tests, giving developers faster feedback.

Which of these is NOT a good practice for parallel tests?

A Sharing global state between tests

B Isolating test data

C Using separate test databases

D Avoiding file conflicts

Sharing global state can cause tests to interfere and fail when run in parallel.

Explain how pytest can run tests in parallel in a CI environment and why this is helpful.

Think about how multiple tests can run at the same time to save time.

Describe one challenge you might face when running tests in parallel and how to avoid it.

Consider what happens if two tests try to change the same file at once.

Practice

1. What is the main benefit of using parallel execution in pytest within a CI environment?

easy

A. It disables flaky tests to improve stability.

B. It automatically fixes failing tests during execution.

C. It generates detailed test coverage reports.

D. It runs multiple tests at the same time to reduce total test time.

Solution

1. Step 1: Understand parallel execution purpose

   Parallel execution means running tests simultaneously instead of one by one.

2. Step 2: Identify benefit in CI context

   Running tests at the same time reduces the total time needed to finish all tests in CI.

3. Final Answer:

   It runs multiple tests at the same time to reduce total test time. -> Option D

4. Quick Check:

   Parallel execution = faster test runs [OK]

Hint: Parallel means multiple tests run together, saving time [OK]

Common Mistakes:

• Confusing parallel execution with automatic bug fixing
• Thinking it generates reports automatically
• Assuming it disables tests instead of running them

2. Which command correctly enables parallel test execution using pytest-xdist with 4 workers?

easy

A. pytest -n 4

B. pytest --workers=4

C. pytest --parallel=4

D. pytest -p xdist 4

Solution

1. Step 1: Recall pytest-xdist syntax

   The pytest-xdist plugin uses the option -n followed by the number of workers.

2. Step 2: Match correct command

   The correct command to run tests in parallel with 4 workers is pytest -n 4.

3. Final Answer:

   pytest -n 4 -> Option A

4. Quick Check:

   Use -n to set worker count [OK]

Hint: Remember: -n sets number of parallel workers [OK]

Common Mistakes:

• Using --workers instead of -n
• Adding number without -n option
• Misplacing plugin name in command

3. Given this pytest command in CI: pytest -n 3 tests/, what is the expected behavior?

medium

A. Tests in the 'tests/' folder run sequentially on one worker.

B. Tests in the 'tests/' folder run in parallel on 3 workers.

C. Only 3 tests will run from the 'tests/' folder.

D. Tests will run with 3 retries on failure.

Solution

1. Step 1: Analyze the command options

   The -n 3 option tells pytest-xdist to use 3 parallel workers.

2. Step 2: Understand test execution effect

   All tests in the 'tests/' folder will be distributed and run simultaneously on 3 workers.

3. Final Answer:

   Tests in the 'tests/' folder run in parallel on 3 workers. -> Option B

4. Quick Check:

   -n 3 means 3 parallel workers [OK]

Hint: -n 3 means run tests on 3 parallel workers [OK]

Common Mistakes:

• Thinking only 3 tests run total
• Assuming tests run sequentially
• Confusing retries with parallelism

4. You added pytest -n 4 in your CI but tests still run sequentially. What is the most likely cause?

medium

A. The tests folder is empty, so no tests run.

B. You need to add --parallel option instead of -n.

C. pytest-xdist plugin is not installed in the CI environment.

D. You must specify the number of retries for parallel to work.

Solution

1. Step 1: Check plugin requirement for parallelism

   pytest-xdist plugin must be installed to enable -n parallel execution.

2. Step 2: Identify cause of sequential runs

   If plugin is missing, pytest ignores -n and runs tests sequentially.

3. Final Answer:

   pytest-xdist plugin is not installed in the CI environment. -> Option C

4. Quick Check:

   Missing plugin causes no parallelism [OK]

Hint: Parallel needs pytest-xdist installed to work [OK]

Common Mistakes:

• Using wrong option like --parallel
• Assuming empty folder causes sequential runs
• Confusing retries with parallel execution

5. In a CI pipeline, you want to run tests in parallel but limit each worker to use only one CPU core to avoid overload. Which pytest-xdist option helps achieve this?

hard

A. Use pytest -n auto --dist loadscope to auto assign workers with load balancing.

B. Use pytest -n 4 --max-worker-threads=1 to limit threads per worker.

C. Use pytest -n 4 --boxed to isolate each test in a subprocess.

D. Use pytest -n 4 --max-worker-memory=1G to limit memory per worker.

Solution

1. Step 1: Understand CPU core limitation in pytest-xdist

   pytest-xdist can auto detect CPU cores and assign workers accordingly using -n auto.

2. Step 2: Use load balancing to distribute tests efficiently

   The --dist loadscope option balances tests to avoid overloading any worker.

3. Final Answer:

   Use pytest -n auto --dist loadscope to auto assign workers with load balancing. -> Option A

4. Quick Check:

   -n auto with loadscope balances CPU load [OK]

Hint: -n auto with --dist loadscope balances CPU load per worker [OK]

Common Mistakes:

• Using non-existent options like --max-worker-threads
• Confusing --boxed with CPU core limits
• Trying to limit memory instead of CPU