Why coverage measures test completeness in PyTest - Challenge Your Understanding

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why is code coverage important in testing?

Code coverage helps us understand how much of our code is tested. Why is this important?

A It measures how fast the tests run.

B It guarantees that all bugs are fixed in the code.

C It replaces the need for writing test cases.

D It shows which parts of the code are executed by tests, helping find untested areas.

💡 Hint

Think about what coverage tells you about your tests and code.

Code coverage shows which parts of the code are run during tests. This helps identify untested code, so you can add tests to cover those parts. It does not guarantee bug fixes or test speed.

❓ Predict Output

intermediate

What coverage percentage is shown after running these tests?

Given this simple Python function and tests, what coverage percentage will pytest-cov report?

PyTest

def add(a, b):
    if a > 0:
        return a + b
    else:
        return b


def test_add_positive():
    assert add(2, 3) == 5

# Note: no test for add with a <= 0

A 75% coverage

B 100% coverage

C 50% coverage

D 25% coverage

💡 Hint

Think about which lines run when add(2, 3) is called.

The test calls add with a=2, so the 'if a > 0' branch runs, but the 'else' branch does not. So some lines are not covered, resulting in about 75% coverage.

❓ assertion

advanced

Which assertion best checks test completeness using coverage data?

You have coverage data showing 85% line coverage. Which assertion best verifies your tests cover at least 80% of lines?

A assert coverage_percent == 85

B assert coverage_percent >= 80

C assert coverage_percent < 80

D assert coverage_percent > 85

💡 Hint

Think about verifying minimum coverage threshold.

To check tests cover at least 80%, assert coverage_percent >= 80. Other options check wrong thresholds or exact values.

🔧 Debug

advanced

Why does this coverage report show 0% for a tested function?

Given this code and test, why does coverage show 0% for the function?

PyTest

def multiply(x, y):
    return x * y


def test_multiply():
    result = multiply(3, 4)
    assert result == 12

# Coverage run command: pytest --cov=module_name
# But coverage shows 0% for multiply function

A The module name in the coverage command is incorrect or missing.

B Coverage only measures test execution time, not code run.

C The multiply function has a syntax error.

D The test assertion is wrong, so coverage is zero.

💡 Hint

Check if coverage is pointed to the right code file.

If coverage is run with a wrong or missing module name, it won't measure the function's code, showing 0% coverage even if tests run.

❓ framework

expert

How to configure pytest-cov to fail tests if coverage is below threshold?

Which pytest-cov option configuration will cause tests to fail if coverage is under 90%?

A pytest --cov=module --coverage-fail=90

B pytest --cov=module --fail-under=90

C pytest --cov=module --cov-fail-under=90

D pytest --cov=module --fail-if-coverage-below=90

💡 Hint

Look for the official pytest-cov option to fail on low coverage.

The correct option is --cov-fail-under=90. Other options are invalid and cause errors.

Practice

1. What does test coverage measure in pytest?

easy

A. How much of the code is executed by tests

B. How many tests are written

C. How fast the tests run

D. How many errors tests find

Solution

1. Step 1: Understand the meaning of coverage

   Coverage shows which parts of the code are run when tests execute.

2. Step 2: Compare options to coverage definition

   Only How much of the code is executed by tests matches this meaning, others describe different test aspects.

3. Final Answer:

   How much of the code is executed by tests -> Option A

4. Quick Check:

   Coverage = executed code percentage [OK]

Hint: Coverage = code run by tests, not test count [OK]

Common Mistakes:

• Confusing coverage with number of tests
• Thinking coverage measures test speed
• Believing coverage counts errors found

2. Which pytest command correctly runs tests with coverage measurement?

easy

A. pytest --cover

B. pytest --coverage

C. pytest -cov-report

D. pytest --cov

Solution

1. Step 1: Recall pytest coverage plugin syntax

   The correct flag to measure coverage is '--cov'.

2. Step 2: Check options for correctness

   Only pytest --cov uses the exact correct flag '--cov'. Others are invalid or incomplete.

3. Final Answer:

   pytest --cov -> Option D

4. Quick Check:

   Use --cov to enable coverage [OK]

Hint: Use '--cov' flag to measure coverage in pytest [OK]

Common Mistakes:

• Using '--coverage' instead of '--cov'
• Mixing coverage report flags with coverage run flags
• Typing '--cover' which is invalid

3. Given this pytest coverage output:

Name          Stmts   Miss  Cover
my_module.py     10      2    80%

What does the 'Miss' number mean?

medium

A. Number of lines not executed by tests

B. Number of errors in code

C. Number of tests skipped

D. Number of tests that failed

Solution

1. Step 1: Understand coverage report columns

   'Miss' shows how many lines of code were not run by tests.

2. Step 2: Match 'Miss' meaning to options

   Number of lines not executed by tests correctly describes 'Miss' as unexecuted lines; others describe unrelated test results.

3. Final Answer:

   Number of lines not executed by tests -> Option A

4. Quick Check:

   Miss = untested lines count [OK]

Hint: 'Miss' means lines tests did not run [OK]

Common Mistakes:

• Thinking 'Miss' counts failed tests
• Confusing 'Miss' with skipped tests
• Assuming 'Miss' means code errors

4. You run pytest with coverage but get 0% coverage report. What is the most likely cause?

medium

A. Tests passed too quickly

B. Tests did not execute any code

C. Coverage plugin is not installed

D. Code has no functions

Solution

1. Step 1: Analyze 0% coverage meaning

   0% coverage means no code lines were run during tests.

2. Step 2: Evaluate causes

   If the coverage plugin is not installed, pytest will not measure coverage and may silently produce 0% coverage report or no coverage data.

3. Final Answer:

   Coverage plugin is not installed -> Option C

4. Quick Check:

   Missing plugin causes no coverage data [OK]

Hint: 0% coverage often means coverage plugin missing [OK]

Common Mistakes:

• Assuming plugin missing causes 0% without errors
• Thinking fast tests mean low coverage
• Believing code without functions can't be covered

5. You want to improve test completeness using coverage. Which approach is best?

hard

A. Write more tests without checking coverage

B. Add tests targeting uncovered code lines shown by coverage report

C. Ignore coverage and focus on test speed

D. Remove tests that run covered code

Solution

1. Step 1: Understand coverage report use

   Coverage shows which code lines lack tests, guiding where to add tests.

2. Step 2: Evaluate options for improving completeness

   Only Add tests targeting uncovered code lines shown by coverage report uses coverage data to add tests for uncovered code, improving completeness.

3. Final Answer:

   Add tests targeting uncovered code lines shown by coverage report -> Option B

4. Quick Check:

   Use coverage to find and test missing code [OK]

Hint: Add tests where coverage report shows gaps [OK]

Common Mistakes:

• Writing tests blindly without coverage info
• Ignoring coverage to focus on speed
• Removing tests that cover code