Generate coverage reports in terminal, HTML, and XML formats using pytest

Preconditions (3)

Step 1: Open a terminal in the project root directory

Step 2: Run pytest with coverage enabled and generate a terminal report using: pytest --cov=your_package

Step 3: Verify the coverage summary is displayed in the terminal

Step 4: Run pytest with coverage enabled and generate an HTML report using: pytest --cov=your_package --cov-report=html

Step 5: Verify an 'htmlcov' directory is created with the HTML report files

Step 6: Open 'htmlcov/index.html' in a browser and verify coverage details are shown

Step 7: Run pytest with coverage enabled and generate an XML report using: pytest --cov=your_package --cov-report=xml

Step 8: Verify a 'coverage.xml' file is created in the project root

Step 9: Open 'coverage.xml' in a text editor and verify it contains XML coverage data

✅ Expected Result: Coverage reports are generated and visible in terminal, HTML, and XML formats as specified

Automation Requirements - pytest with pytest-cov

Assertions Needed:

Verify terminal output contains coverage summary

Verify 'htmlcov/index.html' file exists and contains valid HTML

Verify 'coverage.xml' file exists and contains valid XML coverage data

Best Practices:

Use subprocess module to run pytest commands

Use explicit file existence checks for report files

Parse HTML and XML files to confirm report content validity

Clean up generated report files after test run

Automated Solution

PyTest

import subprocess
import os
import xml.etree.ElementTree as ET
from pathlib import Path


def test_coverage_reports():
    package_name = "your_package"  # Replace with your actual package name

    # Run pytest with coverage and terminal report
    result = subprocess.run(
        ["pytest", f"--cov={package_name}"],
        capture_output=True,
        text=True
    )
    assert result.returncode == 0, "Pytest run failed"
    assert "coverage" in result.stdout.lower(), "Coverage summary not found in terminal output"

    # Run pytest with coverage and HTML report
    result_html = subprocess.run(
        ["pytest", f"--cov={package_name}", "--cov-report=html"],
        capture_output=True,
        text=True
    )
    assert result_html.returncode == 0, "Pytest HTML report run failed"

    html_report_path = Path("htmlcov/index.html")
    assert html_report_path.exists(), "HTML coverage report file does not exist"

    # Simple check that HTML file contains expected tags
    with open(html_report_path, "r", encoding="utf-8") as f:
        html_content = f.read()
    assert "<html" in html_content.lower() and "coverage" in html_content.lower(), "HTML report content invalid"

    # Run pytest with coverage and XML report
    result_xml = subprocess.run(
        ["pytest", f"--cov={package_name}", "--cov-report=xml"],
        capture_output=True,
        text=True
    )
    assert result_xml.returncode == 0, "Pytest XML report run failed"

    xml_report_path = Path("coverage.xml")
    assert xml_report_path.exists(), "XML coverage report file does not exist"

    # Parse XML to verify it is well-formed and contains coverage data
    tree = ET.parse(xml_report_path)
    root = tree.getroot()
    assert root.tag == "coverage", "Root tag of XML report is not 'coverage'"

    # Cleanup generated reports
    if html_report_path.exists():
        for file in html_report_path.parent.glob("**/*"):
            if file.is_file():
                file.unlink()
        html_report_path.parent.rmdir()
    if xml_report_path.exists():
        xml_report_path.unlink()

This test automates the manual steps to generate coverage reports in three formats using pytest and pytest-cov.

First, it runs pytest with coverage enabled and checks the terminal output for coverage summary text.

Next, it runs pytest again to generate an HTML report, then verifies the 'htmlcov/index.html' file exists and contains expected HTML content.

Then, it runs pytest to generate an XML report and parses the 'coverage.xml' file to confirm it is valid XML with the root tag 'coverage'.

Finally, it cleans up the generated report files to keep the project directory clean.

This approach uses subprocess to run commands, file checks for report existence, and content parsing for validation, following best practices for test automation.

Common Mistakes - 4 Pitfalls

Not checking the return code of the pytest subprocess call

Hardcoding package name without clear instruction to replace

Not verifying the content of the HTML or XML reports

Not cleaning up generated report files after test

Bonus Challenge

Now add data-driven testing to generate coverage reports for three different package names

Show Hint