Coverage thresholds in PyTest - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the test result with coverage threshold set to 80%?

Given the following pytest coverage configuration, what will be the test execution result if the actual coverage is 75%?

PyTest

[run]
branch = True

[report]
fail_under = 80

[paths]
source = src

# Assume tests run and coverage measured at 75%

A Tests pass, coverage report shows 75%

B Tests pass but warning about coverage below threshold

C Tests fail due to syntax error in config

D Tests fail due to coverage below 80%

💡 Hint

Coverage fail_under sets minimum coverage percentage to pass tests.

The fail_under setting enforces a minimum coverage percentage. If actual coverage is below 80%, tests fail.

❓ assertion

intermediate

Which assertion correctly checks coverage threshold in pytest?

You want to assert that coverage is at least 90% in a pytest test. Which assertion is correct?

PyTest

def test_coverage(coverage_percent):
    # coverage_percent is a float representing coverage %

A assert coverage_percent > 90

B assert coverage_percent >= 90

C assert coverage_percent == 90

D assert coverage_percent <= 90

💡 Hint

Threshold means coverage must be equal or greater than the set value.

Using >= 90 ensures coverage is at least 90%, matching threshold requirements.

🔧 Debug

advanced

Why does coverage threshold fail even though coverage is 85%?

You set fail_under=85 in your coverage config but tests fail with coverage 85%. What is the likely cause?

PyTest

[report]
fail_under = 85

# Actual coverage measured: 85.0%

A fail_under requires coverage greater or equal to 85%, so failure is unexpected

B fail_under requires coverage strictly greater than 85%

C fail_under is ignored if branch coverage is enabled

D fail_under requires coverage less than 85%

💡 Hint

Check if fail_under is inclusive or exclusive of the threshold value.

fail_under fails the build only if coverage < 85%. Coverage of 85% (>=85%) should pass, so failure is unexpected (e.g., due to floating-point precision, branch coverage, or per-file thresholds).

🧠 Conceptual

advanced

What is the purpose of coverage thresholds in testing?

Why do teams use coverage thresholds in their test suites?

A To ensure a minimum percentage of code is tested to maintain quality

B To guarantee tests run faster by skipping untested code

C To automatically fix uncovered code during tests

D To disable tests if coverage is too high

💡 Hint

Think about quality control in testing.

Coverage thresholds enforce a minimum tested code percentage, helping maintain code quality.

❓ framework

expert

How to configure pytest-cov to fail tests if coverage is below 75%?

Which pytest command line option correctly enforces a 75% coverage threshold that fails tests if not met?

A pytest --cov=src --fail-under=75

B pytest --coverage=src --fail-under=75

C pytest --cov=src --cov-fail-under=75

D pytest --cov=src --cov-fail-over=75

💡 Hint

Check pytest-cov documentation for correct flag names.

The correct flag is --cov-fail-under to set minimum coverage threshold.

Practice

1. What does setting a coverage threshold in pytest ensure?

easy

A. Only tests with coverage above 90% are executed

B. Tests run faster by skipping some checks

C. The test run fails if coverage is below a set percentage

D. Coverage reports are hidden from the user

Solution

1. Step 1: Understand coverage threshold purpose

   Coverage thresholds set a minimum coverage percentage to maintain test quality.

2. Step 2: Effect of coverage below threshold

   If coverage is below the threshold, pytest fails the test run to alert missing tests.

3. Final Answer:

   The test run fails if coverage is below a set percentage -> Option C

4. Quick Check:

   Coverage threshold = fail if below limit [OK]

Hint: Coverage threshold means minimum coverage to pass tests [OK]

Common Mistakes:

• Thinking threshold speeds up tests
• Believing threshold skips tests
• Assuming threshold hides reports

2. Which is the correct pytest command to set a coverage fail threshold at 80%?

easy

A. pytest --cov-fail-under=80

B. pytest --fail-under-cov=80

C. pytest --coverage-threshold=80

D. pytest --fail-coverage=80

Solution

1. Step 1: Recall pytest coverage threshold syntax

   The correct option uses --cov-fail-under to set minimum coverage.

2. Step 2: Match the correct command

   pytest --cov-fail-under=80 matches the exact pytest CLI option for coverage fail threshold.

3. Final Answer:

   pytest --cov-fail-under=80 -> Option A

4. Quick Check:

   Correct CLI option = --cov-fail-under [OK]

Hint: Use --cov-fail-under to set coverage threshold [OK]

Common Mistakes:

• Mixing option order or names
• Using non-existent flags
• Confusing coverage threshold with other options

3. Given this pytest command:
pytest --cov=myapp --cov-fail-under=90
and coverage report shows 85%, what will happen?

medium

A. Tests pass but with a warning

B. Tests fail due to coverage below 90%

C. Tests run without coverage check

D. Coverage report is not generated

Solution

1. Step 1: Understand the coverage threshold set

   The command sets a fail threshold at 90% coverage.

2. Step 2: Compare actual coverage with threshold

   Actual coverage is 85%, which is below 90%, so pytest fails the test run.

3. Final Answer:

   Tests fail due to coverage below 90% -> Option B

4. Quick Check:

   Coverage 85% < 90% threshold = fail [OK]

Hint: Coverage below threshold causes test failure [OK]

Common Mistakes:

• Assuming tests pass with warning
• Thinking coverage check is skipped
• Believing report is not generated

4. You set --cov-fail-under=75 but pytest does not fail even when coverage is 70%. What is the likely cause?

medium

A. You must use --fail-under-cov instead

B. Threshold value must be above 80%

C. Coverage is always ignored in pytest

D. Coverage plugin is not installed or enabled

Solution

1. Step 1: Check coverage plugin status

   If coverage plugin is missing or disabled, threshold has no effect.

2. Step 2: Understand threshold behavior

   Threshold works only if coverage plugin runs and measures coverage.

3. Final Answer:

   Coverage plugin is not installed or enabled -> Option D

4. Quick Check:

   Missing plugin = threshold ignored [OK]

Hint: Ensure coverage plugin is active for thresholds to work [OK]

Common Mistakes:

• Thinking threshold values have minimum limits
• Believing coverage is ignored by default
• Using wrong command option names

5. You want to enforce different coverage thresholds for branches and statements: 80% for branches and 90% for statements. Which pytest configuration snippet achieves this?

hard

A. [run]\nbranch = True\n[report]\nfail_under = 90\nfail_under_branch = 80

B. [run]\nbranch = True\n[report]\nfail_under = 80\nfail_under_branch = 90

C. [coverage]\nbranch = True\nthresholds = statements:90, branches:80

D. [coverage]\nfail_under_statements = 90\nfail_under_branches = 80

Solution

1. Step 1: Enable branch coverage in config

   Set branch = True under [run] to measure branch coverage.

2. Step 2: Set fail thresholds correctly

   Under [report], fail_under sets statement threshold, fail_under_branch sets branch threshold.

3. Final Answer:

   [run]\nbranch = True\n[report]\nfail_under = 90\nfail_under_branch = 80 -> Option A

4. Quick Check:

   Branch coverage enabled + correct thresholds = [run]\nbranch = True\n[report]\nfail_under = 90\nfail_under_branch = 80 [OK]

Hint: Use [run] branch=True and [report] fail_under_branch for branches [OK]

Common Mistakes:

• Swapping statement and branch thresholds
• Using wrong config section names
• Missing branch = True setting