Coverage report formats (terminal, HTML, XML) in PyTest - Test Execution Trace

Test Overview

This test runs pytest with coverage enabled and verifies that coverage reports are generated in terminal, HTML, and XML formats.

Test Code - unittest

PyTest

import subprocess
import os
import unittest

class TestCoverageReports(unittest.TestCase):
    def test_coverage_reports_generation(self):
        # Run pytest with coverage to generate terminal, HTML, and XML reports
        result = subprocess.run([
            'pytest', '--cov=.', '--cov-report=term', '--cov-report=html', '--cov-report=xml'
        ], capture_output=True, text=True)

        # Check pytest run was successful
        self.assertEqual(result.returncode, 0, f"Pytest failed:\n{result.stderr}")

        # Check terminal coverage report output contains coverage summary
        self.assertIn('coverage', result.stdout.lower())

        # Check HTML coverage report directory exists
        self.assertTrue(os.path.isdir('htmlcov'), 'HTML coverage report directory not found')

        # Check XML coverage report file exists
        self.assertTrue(os.path.isfile('coverage.xml'), 'XML coverage report file not found')

if __name__ == '__main__':
    unittest.main()

Execution Trace - 6 Steps

Step	Action	System State	Assertion	Result
1	Test starts	Test runner initialized, no tests running	—	PASS
2	Runs subprocess to execute pytest with coverage options for terminal, HTML, and XML reports	pytest runs tests and collects coverage data	Check subprocess return code is 0 (success)	PASS
3	Reads subprocess stdout to verify terminal coverage report output contains 'coverage'	Terminal output includes coverage summary	Verify 'coverage' keyword present in output	PASS
4	Checks if 'htmlcov' directory exists for HTML coverage report	'htmlcov' directory present with HTML files	Verify 'htmlcov' directory exists	PASS
5	Checks if 'coverage.xml' file exists for XML coverage report	'coverage.xml' file present in current directory	Verify 'coverage.xml' file exists	PASS
6	Test ends successfully	All coverage reports generated and verified	—	PASS

Failure Scenario

Failing Condition: Coverage reports are not generated or pytest run fails

Execution Trace Quiz - 3 Questions

What does the test check after running pytest with coverage options?

A That only terminal coverage report is generated

B That tests fail if coverage is below 100%

C That coverage reports are generated in terminal, HTML, and XML formats

D That coverage reports are saved only in XML format

Key Result

Always verify that coverage reports are generated in all requested formats by checking both the output and the presence of report files or directories.

Practice

1. What is the main purpose of a coverage report in pytest?

easy

A. To generate test data automatically

B. To run tests faster by skipping some tests

C. To fix bugs in the code automatically

D. To show which parts of the code were tested and which were not

Solution

1. Step 1: Understand coverage report purpose

   Coverage reports show which lines or parts of code were executed during tests.

2. Step 2: Compare options with coverage purpose

   Only To show which parts of the code were tested and which were not correctly describes this purpose; others describe unrelated actions.

3. Final Answer:

   To show which parts of the code were tested and which were not -> Option D

4. Quick Check:

   Coverage report = tested code visibility [OK]

Hint: Coverage reports show tested vs untested code [OK]

Common Mistakes:

• Confusing coverage with test execution speed
• Thinking coverage generates test data
• Believing coverage fixes bugs automatically

2. Which pytest command option generates an HTML coverage report?

easy

A. --cov-report=xml

B. --cov-report=summary

C. --cov-report=html

D. --cov-report=term

Solution

1. Step 1: Recall pytest coverage report options

   Common options include 'term' for terminal, 'html' for HTML, and 'xml' for XML reports.

2. Step 2: Match option to HTML report

   Only '--cov-report=html' generates an HTML report.

3. Final Answer:

   --cov-report=html -> Option C

4. Quick Check:

   HTML report option = --cov-report=html [OK]

Hint: HTML report uses --cov-report=html option [OK]

Common Mistakes:

• Using --cov-report=xml for HTML report
• Confusing term with html option
• Using non-existent --cov-report=summary

3. Given this pytest command:
pytest --cov=myapp --cov-report=term-missing
What will the terminal output show?

medium

A. An HTML file opened in the browser

B. A summary of coverage with missing lines shown

C. An XML file saved to disk

D. No coverage information displayed

Solution

1. Step 1: Understand --cov-report=term-missing

   This option shows coverage summary in terminal and highlights missing lines.

2. Step 2: Match output to options

   A summary of coverage with missing lines shown matches terminal summary with missing lines; others describe HTML, XML, or no output.

3. Final Answer:

   A summary of coverage with missing lines shown -> Option B

4. Quick Check:

   term-missing = terminal summary with missing lines [OK]

Hint: term-missing shows missing lines in terminal [OK]

Common Mistakes:

• Thinking term-missing opens HTML report
• Expecting XML output from term-missing
• Assuming no coverage info is shown

4. You ran pytest --cov=myapp --cov-report=html but no HTML report was generated. What is the most likely cause?

medium

A. The HTML report is saved in the current directory as 'htmlcov'

B. You forgot to install the pytest-cov plugin

C. You need to add --cov-report=xml to generate HTML

D. HTML reports are not supported by pytest

Solution

1. Step 1: Check where HTML report is saved

   By default, pytest-cov saves HTML reports in 'htmlcov' folder in current directory.

2. Step 2: Understand common confusion

   Users may think no report generated if they don't check 'htmlcov' folder; plugin installation is needed but question assumes it is installed.

3. Final Answer:

   The HTML report is saved in the current directory as 'htmlcov' -> Option A

4. Quick Check:

   HTML report folder = htmlcov [OK]

Hint: HTML report saved in 'htmlcov' folder by default [OK]

Common Mistakes:

• Assuming HTML report appears in terminal
• Forgetting to look in 'htmlcov' folder
• Thinking XML option is needed for HTML

5. You want to share coverage results with a CI tool that requires XML input. Which pytest command correctly generates the XML coverage report file?

hard

A. pytest --cov=myapp --cov-report=xml

B. pytest --cov=myapp --cov-report=html

C. pytest --cov=myapp --cov-report=term

D. pytest --cov=myapp --cov-report=term-missing

Solution

1. Step 1: Identify XML report option

   The '--cov-report=xml' option generates coverage results in XML format.

2. Step 2: Match option to CI tool requirement

   CI tools needing XML input require this exact option; others generate HTML or terminal output.

3. Final Answer:

   pytest --cov=myapp --cov-report=xml -> Option A

4. Quick Check:

   XML report option = --cov-report=xml [OK]

Hint: Use --cov-report=xml for CI XML coverage [OK]

Common Mistakes:

• Using HTML or terminal options for XML output
• Not specifying any --cov-report option
• Confusing term-missing with XML