Bird
Raised Fist0
PyTesttesting~15 mins

Why coverage measures test completeness in PyTest - Automation Benefits in Action

Choose your learning style10 modes available

Start learning this pattern below

Jump into concepts and practice - no test required

or
Recommended
Test this pattern10 questions across easy, medium, and hard to know if this pattern is strong
Verify code coverage reports test completeness
Preconditions (3)
Step 1: Create a Python function that returns the square of a number
Step 2: Write a pytest test function that tests the square function with one input
Step 3: Run pytest with coverage enabled to measure coverage
Step 4: Check the coverage report to see which lines were executed
Step 5: Add another test case to cover missing lines
Step 6: Run pytest with coverage again
Step 7: Verify that coverage percentage increases and reaches 100%
✅ Expected Result: Coverage report shows initial partial coverage, then full coverage after adding tests, demonstrating test completeness
Automation Requirements - pytest with coverage
Assertions Needed:
Assert that coverage percentage is less than 100% after first test run
Assert that coverage percentage is 100% after adding all test cases
Best Practices:
Use pytest fixtures for setup if needed
Use coverage.py command line options to generate reports
Keep tests small and focused
Use assert statements to verify function output
Automated Solution
PyTest
import subprocess
import re

def square(x):
    return x * x

def test_square_one():
    assert square(3) == 9

# Run coverage for initial test
result1 = subprocess.run(['coverage', 'run', '-m', 'pytest', '-q', '--tb=line'], capture_output=True, text=True)
result_report1 = subprocess.run(['coverage', 'report'], capture_output=True, text=True)

# Extract coverage percent for the file
match1 = re.search(r'square.py\s+\d+\s+\d+\s+(\d+)%', result_report1.stdout)
coverage1 = int(match1.group(1)) if match1 else 0

# Add second test to cover all lines

def test_square_zero():
    assert square(0) == 0

# Run coverage again with both tests
result2 = subprocess.run(['coverage', 'run', '-m', 'pytest', '-q', '--tb=line'], capture_output=True, text=True)
result_report2 = subprocess.run(['coverage', 'report'], capture_output=True, text=True)

match2 = re.search(r'square.py\s+\d+\s+\d+\s+(\d+)%', result_report2.stdout)
coverage2 = int(match2.group(1)) if match2 else 0

# Assertions
assert coverage1 < 100, f"Expected coverage less than 100%, got {coverage1}%"
assert coverage2 == 100, f"Expected coverage 100%, got {coverage2}%"

This script defines a simple function square and two pytest test functions.

First, it runs pytest with coverage enabled with only one test, then captures the coverage report and extracts the coverage percentage.

Then it adds a second test to cover all code paths and runs coverage again.

Finally, it asserts that the first coverage is less than 100% and the second coverage is exactly 100%, showing how coverage measures test completeness.

Using subprocess allows running coverage commands programmatically.

Regex extracts coverage percent from the report output.

This approach teaches how coverage helps identify missing tests and improve completeness.

Common Mistakes - 3 Pitfalls
{'mistake': 'Not running coverage with the correct pytest command', 'why_bad': 'Coverage will not measure the tests properly if pytest is not run with coverage enabled', 'correct_approach': "Always run tests using 'coverage run -m pytest' to collect coverage data"}
Hardcoding coverage percentage values in assertions
Not adding tests to cover all code paths
Bonus Challenge

Now add data-driven testing with 3 different inputs to the square function

Show Hint

Practice

(1/5)
1. What does test coverage measure in pytest?
easy
A. How much of the code is executed by tests
B. How many tests are written
C. How fast the tests run
D. How many errors tests find

Solution

  1. Step 1: Understand the meaning of coverage

    Coverage shows which parts of the code are run when tests execute.
  2. Step 2: Compare options to coverage definition

    Only How much of the code is executed by tests matches this meaning, others describe different test aspects.
  3. Final Answer:

    How much of the code is executed by tests -> Option A
  4. Quick Check:

    Coverage = executed code percentage [OK]
Hint: Coverage = code run by tests, not test count [OK]
Common Mistakes:
  • Confusing coverage with number of tests
  • Thinking coverage measures test speed
  • Believing coverage counts errors found
2. Which pytest command correctly runs tests with coverage measurement?
easy
A. pytest --cover
B. pytest --coverage
C. pytest -cov-report
D. pytest --cov

Solution

  1. Step 1: Recall pytest coverage plugin syntax

    The correct flag to measure coverage is '--cov'.
  2. Step 2: Check options for correctness

    Only pytest --cov uses the exact correct flag '--cov'. Others are invalid or incomplete.
  3. Final Answer:

    pytest --cov -> Option D
  4. Quick Check:

    Use --cov to enable coverage [OK]
Hint: Use '--cov' flag to measure coverage in pytest [OK]
Common Mistakes:
  • Using '--coverage' instead of '--cov'
  • Mixing coverage report flags with coverage run flags
  • Typing '--cover' which is invalid
3. Given this pytest coverage output:
Name          Stmts   Miss  Cover
my_module.py     10      2    80%

What does the 'Miss' number mean?
medium
A. Number of lines not executed by tests
B. Number of errors in code
C. Number of tests skipped
D. Number of tests that failed

Solution

  1. Step 1: Understand coverage report columns

    'Miss' shows how many lines of code were not run by tests.
  2. Step 2: Match 'Miss' meaning to options

    Number of lines not executed by tests correctly describes 'Miss' as unexecuted lines; others describe unrelated test results.
  3. Final Answer:

    Number of lines not executed by tests -> Option A
  4. Quick Check:

    Miss = untested lines count [OK]
Hint: 'Miss' means lines tests did not run [OK]
Common Mistakes:
  • Thinking 'Miss' counts failed tests
  • Confusing 'Miss' with skipped tests
  • Assuming 'Miss' means code errors
4. You run pytest with coverage but get 0% coverage report. What is the most likely cause?
medium
A. Tests passed too quickly
B. Tests did not execute any code
C. Coverage plugin is not installed
D. Code has no functions

Solution

  1. Step 1: Analyze 0% coverage meaning

    0% coverage means no code lines were run during tests.
  2. Step 2: Evaluate causes

    If the coverage plugin is not installed, pytest will not measure coverage and may silently produce 0% coverage report or no coverage data.
  3. Final Answer:

    Coverage plugin is not installed -> Option C
  4. Quick Check:

    Missing plugin causes no coverage data [OK]
Hint: 0% coverage often means coverage plugin missing [OK]
Common Mistakes:
  • Assuming plugin missing causes 0% without errors
  • Thinking fast tests mean low coverage
  • Believing code without functions can't be covered
5. You want to improve test completeness using coverage. Which approach is best?
hard
A. Write more tests without checking coverage
B. Add tests targeting uncovered code lines shown by coverage report
C. Ignore coverage and focus on test speed
D. Remove tests that run covered code

Solution

  1. Step 1: Understand coverage report use

    Coverage shows which code lines lack tests, guiding where to add tests.
  2. Step 2: Evaluate options for improving completeness

    Only Add tests targeting uncovered code lines shown by coverage report uses coverage data to add tests for uncovered code, improving completeness.
  3. Final Answer:

    Add tests targeting uncovered code lines shown by coverage report -> Option B
  4. Quick Check:

    Use coverage to find and test missing code [OK]
Hint: Add tests where coverage report shows gaps [OK]
Common Mistakes:
  • Writing tests blindly without coverage info
  • Ignoring coverage to focus on speed
  • Removing tests that cover code