JUnit XML reporting for CI in PyTest - Build an Automation Script
Start learning this pattern below
Jump into concepts and practice - no test required
import subprocess import os import xml.etree.ElementTree as ET import unittest class TestPytestJUnitReport(unittest.TestCase): def test_generate_junit_xml_report(self): # Run pytest with JUnit XML output result = subprocess.run(['pytest', '--junitxml=report.xml', 'test_sample.py'], capture_output=True, text=True) # Assert pytest ran successfully self.assertEqual(result.returncode, 0, f"pytest failed with output:\n{result.stdout}\n{result.stderr}") # Assert report.xml file exists self.assertTrue(os.path.exists('report.xml'), 'report.xml file was not created') # Parse XML and check for testsuite and testcase elements tree = ET.parse('report.xml') root = tree.getroot() self.assertEqual(root.tag, 'testsuite', 'Root element is not <testsuite>') testcases = root.findall('testcase') self.assertGreater(len(testcases), 0, 'No <testcase> elements found in report.xml') if __name__ == '__main__': unittest.main()
This script uses Python's unittest framework to automate the manual test case steps.
First, it runs pytest with the --junitxml=report.xml option using subprocess.run. This generates the JUnit XML report file.
Then, it checks the return code to ensure pytest ran successfully.
Next, it verifies that the report.xml file was created in the current directory.
Finally, it parses the XML file using xml.etree.ElementTree to confirm the root element is <testsuite> and that there is at least one <testcase> element inside, ensuring the report is valid and contains test results.
This approach follows best practices by using subprocess for running commands, checking outputs, and parsing XML to validate the report content.
Now add data-driven testing to run pytest with different test files and generate separate JUnit XML reports for each.
Practice
Solution
Step 1: Understand JUnit XML report role
JUnit XML reports provide a standard format for test results that many CI tools can parse and display.Step 2: Compare options with purpose
Speeding tests, fixing tests, or generating HTML are not the main goals of JUnit XML reporting.Final Answer:
To provide a standard format for test results that CI tools can read -> Option AQuick Check:
JUnit XML = standard test result format [OK]
- Thinking JUnit XML speeds up tests
- Confusing XML report with HTML report
- Assuming it fixes tests automatically
results.xml?Solution
Step 1: Recall pytest option for JUnit XML
The correct option to generate JUnit XML report is--junitxml=filename.Step 2: Match option with command
Only pytest --junitxml=results.xml uses--junitxml=results.xml, which is correct syntax.Final Answer:
pytest --junitxml=results.xml -> Option DQuick Check:
--junitxml option = correct flag [OK]
- Using --xml instead of --junitxml
- Using --report or --output which are invalid
- Missing the equals sign '='
pytest tests/ --junitxml=report.xmlWhat will happen after running it?
Solution
Step 1: Understand pytest command behavior
The command runs tests in thetests/folder and generates a JUnit XML report file namedreport.xml.Step 2: Check output expectations
Since--junitxml=report.xmlis specified, the file will be created with test results in XML format.Final Answer:
Tests run and a file named report.xml is created with test results in JUnit XML format -> Option BQuick Check:
--junitxml creates XML report file [OK]
- Thinking tests don't run when report option is used
- Expecting report file without running tests
- Assuming output only prints to console
pytest --junitxml=results.xml but the file results.xml is empty. What is the most likely cause?Solution
Step 1: Analyze empty report cause
An empty JUnit XML file usually means no tests were found or executed.Step 2: Check other options
The option--junitxmlis correct, pytest supports it, and invalid paths usually cause errors, not silent empty files.Final Answer:
No tests were collected or run -> Option AQuick Check:
Empty XML = no tests run [OK]
- Assuming option is misspelled without checking
- Ignoring that no tests might be collected
- Thinking pytest doesn't support JUnit XML
ci-report.xml. Which command should you use?Solution
Step 1: Understand options for failing fast
To stop on first failure and fail the build, use--failfastand limit failures with--maxfail=1.Step 2: Confirm JUnit XML report option
--junitxml=ci-report.xmlsaves the report file as required.Step 3: Evaluate options
pytest --junitxml=ci-report.xml --failfast --maxfail=1 correctly combines--junitxml=ci-report.xml,--failfast, and--maxfail=1. Others have wrong or conflicting flags.Final Answer:
pytest --junitxml=ci-report.xml --failfast --maxfail=1 -> Option CQuick Check:
Fail fast + maxfail=1 + junitxml = pytest --junitxml=ci-report.xml --failfast --maxfail=1 [OK]
- Confusing --exitfirst with --failfast
- Setting maxfail=0 disables failure limit
- Omitting --failfast causes tests to run fully
