Coverage in CI pipelines in PyTest - Test Execution Trace

Test Overview

This test runs a Python pytest suite with coverage measurement in a Continuous Integration (CI) pipeline. It verifies that the test coverage meets the minimum required threshold.

Test Code - pytest

PyTest

import pytest
import subprocess

def test_coverage_threshold():
    # Run pytest with coverage and generate a coverage report
    result = subprocess.run([
        'pytest', '--cov=.', '--cov-report=term-missing', '--cov-fail-under=80', '-k', 'not test_coverage_threshold'
    ], capture_output=True, text=True)

    # Assert that pytest exited with code 0 (success)
    assert result.returncode == 0, f"Coverage threshold not met. Output:\n{result.stdout}\n{result.stderr}"

Execution Trace - 5 Steps

Step	Action	System State	Assertion	Result
1	Test starts: pytest command with coverage options is executed	CI environment runs pytest with coverage measuring all source files	-	PASS
2	pytest discovers and runs all test functions	Tests execute, coverage data is collected for each source line	-	PASS
3	pytest generates coverage report showing lines covered and missing	Coverage report output is available in terminal	-	PASS
4	pytest checks if coverage percentage meets or exceeds 80%	Coverage threshold check performed internally by pytest-cov plugin	Assert that pytest exit code is 0 indicating coverage threshold met	PASS
5	Test ends with success if coverage threshold is met	CI pipeline receives success status	-	PASS

Failure Scenario

Failing Condition: Coverage is below 80%, causing pytest to exit with non-zero code

Execution Trace Quiz - 3 Questions

What does the test assert to verify coverage in the CI pipeline?

A That all tests pass regardless of coverage

B That coverage report file exists on disk

C That pytest exits with code 0 indicating coverage threshold met

D That coverage percentage is exactly 100%

Key Result

Always enforce a minimum coverage threshold in CI pipelines to catch untested code early and maintain code quality.