Coverage in CI pipelines in PyTest - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Understanding coverage report output in pytest

What is the output of the following pytest coverage command snippet when run on a module with 3 functions, where 2 functions are tested and 1 is not?

PyTest

pytest --cov=my_module --cov-report=term

# Assume my_module has 3 functions: func_a, func_b, func_c
# Tests cover func_a and func_b only

A Coverage: 0% of statements executed

B Coverage: 66% of statements executed

C Coverage: 100% of statements executed

D Coverage: 33% of statements executed

💡 Hint

Think about how coverage percentage is calculated based on tested functions.

Since 2 out of 3 functions are tested, coverage is approximately 66%.

❓ assertion

intermediate

Correct assertion for coverage threshold in pytest

Which assertion correctly fails the test if coverage is below 80% in a pytest coverage report?

PyTest

coverage_percent = 75  # example value from coverage report

# Choose the correct assertion

A assert coverage_percent >= 80, f"Coverage too low: {coverage_percent}%"

B assert coverage_percent <= 80, f"Coverage too low: {coverage_percent}%"

C assert coverage_percent > 80, f"Coverage too low: {coverage_percent}%"

D assert coverage_percent < 80, f"Coverage too low: {coverage_percent}%"

💡 Hint

Coverage should be at least 80%.

Assertion B correctly checks coverage is 80% or more, failing otherwise.

❓ locator

advanced

Best practice for locating coverage report file in CI

In a CI pipeline, where should the coverage report file be stored for best accessibility and integration?

A Inside the source code directory alongside the code files

B In a temporary system directory outside the project

C In a dedicated reports directory at the root of the project

D In the user's home directory on the CI server

💡 Hint

Think about organization and ease of access for CI tools.

Storing reports in a dedicated directory at the project root keeps files organized and accessible.

❓ framework

advanced

Integrating coverage checks in pytest CI pipeline

Which pytest command line option combination ensures tests fail if coverage is below 90% during CI runs?

A pytest --cov=my_module --coverage-fail=90

B pytest --cov=my_module --fail-under=90

C pytest --coverage=my_module --fail-under=90

D pytest --cov=my_module --cov-fail-under=90

💡 Hint

Look for the correct pytest coverage fail threshold option.

The option --cov-fail-under=90 makes pytest fail if coverage is below 90%.

🔧 Debug

expert

Debugging missing coverage data in CI pipeline

In a CI pipeline, coverage reports show 0% coverage despite tests running successfully. What is the most likely cause?

A Coverage data file is overwritten before report generation

B Tests are not importing the module under test

C Coverage tool is not installed in the CI environment

D Coverage measurement started after tests finished running

💡 Hint

Think about how coverage data files are handled in CI.

If coverage data files are overwritten or deleted before report generation, coverage shows 0%.

Practice

1. What is the main purpose of using coverage tools like pytest --cov in CI pipelines?

easy

A. To generate user documentation

B. To speed up the test execution time

C. To deploy the application automatically

D. To measure how much of the code is tested automatically

Solution

1. Step 1: Understand coverage tools in testing

   Coverage tools measure the percentage of code executed by tests.

2. Step 2: Role in CI pipelines

   In CI, coverage helps ensure tests cover enough code to catch bugs early.

3. Final Answer:

   To measure how much of the code is tested automatically -> Option D

4. Quick Check:

   Coverage measures tested code = D [OK]

Hint: Coverage shows tested code percentage in CI [OK]

Common Mistakes:

• Confusing coverage with test speed
• Thinking coverage deploys code
• Assuming coverage creates docs

2. Which of the following is the correct command to run pytest with coverage reporting?

easy

A. pytest --cov=.

B. pytest --cov-report=html --cov

C. pytest -coverage

D. pytest --coverage=report

Solution

1. Step 1: Identify correct pytest coverage syntax

   The correct syntax uses --cov to specify coverage on the current directory.

2. Step 2: Analyze options

   pytest --cov=. correctly runs coverage on the current directory with default terminal reporting.

3. Final Answer:

   pytest --cov=. -> Option A

4. Quick Check:

   Correct pytest coverage command = B [OK]

Hint: Use --cov and --cov-report together for coverage output [OK]

Common Mistakes:

• Using incorrect flags like --coverage=report
• Missing --cov option
• Wrong flag syntax like -coverage

3. Given this pytest command in a CI pipeline:

pytest --cov=myapp --cov-report=term

What will be the output shown in the CI logs?

medium

A. A summary of coverage percentages printed in the terminal

B. No coverage information will be shown

C. A detailed HTML coverage report saved to disk

D. An error because --cov-report=term is invalid

Solution

1. Step 1: Understand --cov-report=term option

   The term option prints coverage summary in the terminal output.

2. Step 2: Analyze expected CI log output

   The CI logs will show coverage percentages summary, not an HTML file or error.

3. Final Answer:

   A summary of coverage percentages printed in the terminal -> Option A

4. Quick Check:

   --cov-report=term shows summary in terminal = C [OK]

Hint: term report prints coverage summary in console [OK]

Common Mistakes:

• Expecting HTML report in terminal
• Assuming no coverage output
• Thinking --cov-report=term is invalid

4. You added coverage to your CI pipeline with this command:

pytest --cov=myapp --cov-report=html

But the coverage report is missing after the run. What is the most likely cause?

medium

A. The --cov option is misspelled

B. The coverage report is saved in a different directory, not checked by CI

C. pytest does not support coverage reporting

D. The tests did not run because of a syntax error

Solution

1. Step 1: Understand where coverage HTML reports are saved

   By default, HTML reports are saved in a folder named htmlcov in the current directory.

2. Step 2: Check CI pipeline file handling

   If the CI pipeline does not collect or upload this folder, the report will appear missing.

3. Final Answer:

   The coverage report is saved in a different directory, not checked by CI -> Option B

4. Quick Check:

   HTML report saved in htmlcov folder = A [OK]

Hint: Check htmlcov folder location in CI artifacts [OK]

Common Mistakes:

• Assuming --cov is misspelled without checking
• Believing pytest lacks coverage support
• Ignoring test run errors

5. In a CI pipeline, you want to fail the build if coverage falls below 80%. Which pytest coverage option helps enforce this?

hard

A. --fail-if-coverage-below=80

B. --coverage-threshold=80

C. --cov-fail-under=80

D. --cov-minimum=80

Solution

1. Step 1: Identify pytest coverage option for minimum coverage

   The correct option is --cov-fail-under which sets a minimum coverage percentage.

2. Step 2: Understand CI build failure behavior

   If coverage is below the set value, pytest returns a failure status causing CI to fail the build.

3. Final Answer:

   --cov-fail-under=80 -> Option C

4. Quick Check:

   Fail build if coverage under 80% = B [OK]

Hint: Use --cov-fail-under to enforce coverage minimum [OK]

Common Mistakes:

• Using incorrect or non-existent flags
• Confusing coverage reporting with build failure
• Assuming coverage threshold is set elsewhere