Why coverage measures test completeness in PyTest - Automation Benefits in Action

Verify code coverage reports test completeness

Preconditions (3)

Step 1: Create a Python function that returns the square of a number

Step 2: Write a pytest test function that tests the square function with one input

Step 3: Run pytest with coverage enabled to measure coverage

Step 4: Check the coverage report to see which lines were executed

Step 5: Add another test case to cover missing lines

Step 6: Run pytest with coverage again

Step 7: Verify that coverage percentage increases and reaches 100%

✅ Expected Result: Coverage report shows initial partial coverage, then full coverage after adding tests, demonstrating test completeness

Automation Requirements - pytest with coverage

Assertions Needed:

Assert that coverage percentage is less than 100% after first test run

Assert that coverage percentage is 100% after adding all test cases

Best Practices:

Use pytest fixtures for setup if needed

Use coverage.py command line options to generate reports

Keep tests small and focused

Use assert statements to verify function output

Automated Solution

PyTest

import subprocess
import re

def square(x):
    return x * x

def test_square_one():
    assert square(3) == 9

# Run coverage for initial test
result1 = subprocess.run(['coverage', 'run', '-m', 'pytest', '-q', '--tb=line'], capture_output=True, text=True)
result_report1 = subprocess.run(['coverage', 'report'], capture_output=True, text=True)

# Extract coverage percent for the file
match1 = re.search(r'square.py\s+\d+\s+\d+\s+(\d+)%', result_report1.stdout)
coverage1 = int(match1.group(1)) if match1 else 0

# Add second test to cover all lines

def test_square_zero():
    assert square(0) == 0

# Run coverage again with both tests
result2 = subprocess.run(['coverage', 'run', '-m', 'pytest', '-q', '--tb=line'], capture_output=True, text=True)
result_report2 = subprocess.run(['coverage', 'report'], capture_output=True, text=True)

match2 = re.search(r'square.py\s+\d+\s+\d+\s+(\d+)%', result_report2.stdout)
coverage2 = int(match2.group(1)) if match2 else 0

# Assertions
assert coverage1 < 100, f"Expected coverage less than 100%, got {coverage1}%"
assert coverage2 == 100, f"Expected coverage 100%, got {coverage2}%"

This script defines a simple function square and two pytest test functions.

First, it runs pytest with coverage enabled with only one test, then captures the coverage report and extracts the coverage percentage.

Then it adds a second test to cover all code paths and runs coverage again.

Finally, it asserts that the first coverage is less than 100% and the second coverage is exactly 100%, showing how coverage measures test completeness.

Using subprocess allows running coverage commands programmatically.

Regex extracts coverage percent from the report output.

This approach teaches how coverage helps identify missing tests and improve completeness.

Common Mistakes - 3 Pitfalls

{'mistake': 'Not running coverage with the correct pytest command', 'why_bad': 'Coverage will not measure the tests properly if pytest is not run with coverage enabled', 'correct_approach': "Always run tests using 'coverage run -m pytest' to collect coverage data"}

Hardcoding coverage percentage values in assertions

Not adding tests to cover all code paths

Bonus Challenge

Now add data-driven testing with 3 different inputs to the square function

Show Hint