pytest-cov for coverage - Build an Automation Script

Measure test coverage using pytest-cov

Preconditions (4)

Step 1: Run pytest with the pytest-cov plugin enabled to measure coverage of calculator.py

Step 2: Use the command: pytest --cov=calculator test_calculator.py

Step 3: Observe the coverage report printed in the console

✅ Expected Result: The console shows a coverage report indicating the percentage of calculator.py code covered by tests, with no errors or failures

Automation Requirements - pytest

Assertions Needed:

Verify that the coverage report includes calculator.py

Verify that coverage percentage is 100% for calculator.py

Best Practices:

Use pytest-cov command line options correctly

Keep test and source code separate

Use clear and descriptive test function names

Automated Solution

PyTest

import subprocess
import re

def test_coverage_report():
    # Run pytest with coverage on calculator.py
    result = subprocess.run(
        ['pytest', '--cov=calculator', 'test_calculator.py', '--cov-report=term'],
        capture_output=True,
        text=True
    )
    output = result.stdout

    # Check pytest ran successfully
    assert result.returncode == 0, f"Pytest failed with output:\n{output}"

    # Check coverage report contains calculator.py
    assert 'calculator.py' in output, "Coverage report does not include calculator.py"

    # Extract coverage percentage for calculator.py using regex
    match = re.search(r'calculator.py\s+\d+\s+\d+\s+(\d+)%', output)
    assert match is not None, "Could not find coverage percentage for calculator.py"

    coverage_percent = int(match.group(1))
    assert coverage_percent == 100, f"Expected 100% coverage but got {coverage_percent}%"

This test runs pytest as a subprocess with the --cov=calculator option to measure coverage of the calculator.py module.

It captures the output and checks that pytest completed successfully (return code 0).

Then it verifies the coverage report includes calculator.py and extracts the coverage percentage using a regular expression.

Finally, it asserts the coverage is 100%, meaning all code lines in calculator.py are tested.

This approach automates verifying that coverage measurement works as expected using pytest-cov.

Common Mistakes - 3 Pitfalls

{'mistake': 'Not installing pytest-cov plugin before running coverage', 'why_bad': "Without pytest-cov installed, coverage options won't work and tests won't measure coverage", 'correct_approach': 'Install pytest-cov using pip before running coverage commands'}

Running pytest without specifying the module for coverage

Parsing coverage output incorrectly or not checking return code

Bonus Challenge

Now add data-driven testing to verify coverage for multiple modules like calculator.py and utils.py

Show Hint

Practice

1. What is the main purpose of using pytest-cov in your testing workflow?

easy

A. To generate test data automatically

B. To speed up test execution time

C. To automatically fix failing tests

D. To measure how much of your code is covered by tests

Solution

1. Step 1: Understand pytest-cov functionality

   pytest-cov is a plugin that measures code coverage during test runs.

2. Step 2: Identify the main goal of coverage tools

   Coverage tools show which parts of code are tested and which are not.

3. Final Answer:

   To measure how much of your code is covered by tests -> Option D

4. Quick Check:

   pytest-cov = code coverage measurement [OK]

Hint: Coverage means how much code tests run over [OK]

Common Mistakes:

• Confusing coverage with test speed
• Thinking pytest-cov fixes tests automatically
• Assuming pytest-cov generates test data

2. Which command correctly runs pytest with coverage for a package named myapp?

easy

A. pytest -cov myapp

B. pytest --cov=myapp

C. pytest --coverage myapp

D. pytest --cov tests/

Solution

1. Step 1: Recall correct pytest-cov syntax

   The correct option to enable coverage is --cov=package_name.

2. Step 2: Compare options

   pytest --cov=myapp uses --cov=myapp, which is the correct syntax. pytest --cov tests/ would incorrectly target the tests directory instead of the myapp package.

3. Final Answer:

   pytest --cov=myapp -> Option B

4. Quick Check:

   Use --cov=package [OK]

Hint: Use equal sign with --cov like --cov=package [OK]

Common Mistakes:

• Targeting tests directory instead of the package
• Using -cov instead of --cov
• Using --coverage instead of --cov

3. Given this command run:
pytest --cov=myapp tests/test_module.py
What output will you expect related to coverage?

medium

A. A report showing coverage percentage for myapp package

B. A list of all tests run without coverage info

C. An error saying coverage option is invalid

D. Coverage report for tests directory only

Solution

1. Step 1: Understand coverage target

   The --cov=myapp option measures coverage for the myapp package, not tests.

2. Step 2: Predict output

   Running tests in tests/test_module.py with coverage enabled will show coverage report for myapp code exercised by those tests.

3. Final Answer:

   A report showing coverage percentage for myapp package -> Option A

4. Quick Check:

   --cov=package shows coverage for that package [OK]

Hint: Coverage reports show tested code, not test files [OK]

Common Mistakes:

• Expecting coverage for test files instead of source code
• Thinking coverage option causes errors
• Confusing coverage report with test list output

4. You ran pytest --cov=myapp but no coverage report appeared. What is the most likely cause?

medium

A. You forgot to install pytest-cov plugin

B. You ran pytest without any test files

C. Your code has no functions to cover

D. You used wrong package name in --cov option

Solution

1. Step 1: Check plugin installation

   pytest-cov must be installed to enable coverage reporting.

2. Step 2: Understand symptoms

   If no coverage report appears, the plugin is likely missing or not activated.

3. Final Answer:

   You forgot to install pytest-cov plugin -> Option A

4. Quick Check:

   pytest-cov missing = no coverage report [OK]

Hint: Install pytest-cov to get coverage reports [OK]

Common Mistakes:

• Assuming coverage works without plugin installed
• Thinking coverage depends on code content
• Ignoring plugin installation errors

5. You want to generate an HTML coverage report for your package app using pytest-cov. Which command should you run?

hard

A. pytest --coverage=app --report=html

B. pytest --cov=app --html=coverage

C. pytest --cov=app --cov-report=html

D. pytest --cov app --html-report

Solution

1. Step 1: Recall pytest-cov report syntax

   The option --cov-report=html generates an HTML coverage report.

2. Step 2: Match correct command format

   pytest --cov=app --cov-report=html correctly uses --cov=app and --cov-report=html together.

3. Final Answer:

   pytest --cov=app --cov-report=html -> Option C

4. Quick Check:

   Use --cov-report=html for HTML coverage [OK]

Hint: Use --cov-report=html to get HTML coverage output [OK]

Common Mistakes:

• Using --html instead of --cov-report=html
• Omitting equal signs in options
• Confusing coverage options with test report options