Bird
Raised Fist0
PyTesttesting~15 mins

JUnit XML reporting for CI in PyTest - Build an Automation Script

Choose your learning style10 modes available

Start learning this pattern below

Jump into concepts and practice - no test required

or
Recommended
Test this pattern10 questions across easy, medium, and hard to know if this pattern is strong
Generate JUnit XML report using pytest for CI integration
Preconditions (2)
Step 1: Open terminal or command prompt
Step 2: Navigate to the directory containing test_sample.py
Step 3: Run pytest with the option to generate JUnit XML report: pytest --junitxml=report.xml
Step 4: Verify that the report.xml file is created in the current directory
Step 5: Open report.xml and check it contains test suite and test case information in XML format
✅ Expected Result: pytest runs tests successfully and creates a valid JUnit XML report file named report.xml that can be used by CI tools
Automation Requirements - pytest
Assertions Needed:
Verify pytest command runs without errors
Verify report.xml file is created after test run
Verify report.xml contains valid XML with <testsuite> and <testcase> elements
Best Practices:
Use subprocess module to run pytest command
Use xml.etree.ElementTree to parse and validate XML content
Check file existence and content programmatically
Keep test code clean and readable
Automated Solution
PyTest
import subprocess
import os
import xml.etree.ElementTree as ET
import unittest

class TestPytestJUnitReport(unittest.TestCase):
    def test_generate_junit_xml_report(self):
        # Run pytest with JUnit XML output
        result = subprocess.run(['pytest', '--junitxml=report.xml', 'test_sample.py'], capture_output=True, text=True)
        
        # Assert pytest ran successfully
        self.assertEqual(result.returncode, 0, f"pytest failed with output:\n{result.stdout}\n{result.stderr}")
        
        # Assert report.xml file exists
        self.assertTrue(os.path.exists('report.xml'), 'report.xml file was not created')
        
        # Parse XML and check for testsuite and testcase elements
        tree = ET.parse('report.xml')
        root = tree.getroot()
        self.assertEqual(root.tag, 'testsuite', 'Root element is not <testsuite>')
        testcases = root.findall('testcase')
        self.assertGreater(len(testcases), 0, 'No <testcase> elements found in report.xml')

if __name__ == '__main__':
    unittest.main()

This script uses Python's unittest framework to automate the manual test case steps.

First, it runs pytest with the --junitxml=report.xml option using subprocess.run. This generates the JUnit XML report file.

Then, it checks the return code to ensure pytest ran successfully.

Next, it verifies that the report.xml file was created in the current directory.

Finally, it parses the XML file using xml.etree.ElementTree to confirm the root element is <testsuite> and that there is at least one <testcase> element inside, ensuring the report is valid and contains test results.

This approach follows best practices by using subprocess for running commands, checking outputs, and parsing XML to validate the report content.

Common Mistakes - 4 Pitfalls
Not checking the pytest command return code
Hardcoding file paths without verifying current directory
Parsing XML without error handling
Not verifying the content of the XML report
Bonus Challenge

Now add data-driven testing to run pytest with different test files and generate separate JUnit XML reports for each.

Show Hint

Practice

(1/5)
1. What is the main purpose of generating a JUnit XML report in pytest for CI pipelines?
easy
A. To provide a standard format for test results that CI tools can read
B. To speed up test execution time
C. To automatically fix failing tests
D. To generate HTML reports for manual review

Solution

  1. Step 1: Understand JUnit XML report role

    JUnit XML reports provide a standard format for test results that many CI tools can parse and display.
  2. Step 2: Compare options with purpose

    Speeding tests, fixing tests, or generating HTML are not the main goals of JUnit XML reporting.
  3. Final Answer:

    To provide a standard format for test results that CI tools can read -> Option A
  4. Quick Check:

    JUnit XML = standard test result format [OK]
Hint: JUnit XML = standard test results for CI tools [OK]
Common Mistakes:
  • Thinking JUnit XML speeds up tests
  • Confusing XML report with HTML report
  • Assuming it fixes tests automatically
2. Which pytest command correctly generates a JUnit XML report named results.xml?
easy
A. pytest --xml=results.xml
B. pytest --output=results.xml
C. pytest --report=results.xml
D. pytest --junitxml=results.xml

Solution

  1. Step 1: Recall pytest option for JUnit XML

    The correct option to generate JUnit XML report is --junitxml=filename.
  2. Step 2: Match option with command

    Only pytest --junitxml=results.xml uses --junitxml=results.xml, which is correct syntax.
  3. Final Answer:

    pytest --junitxml=results.xml -> Option D
  4. Quick Check:

    --junitxml option = correct flag [OK]
Hint: Use --junitxml=filename.xml to generate report [OK]
Common Mistakes:
  • Using --xml instead of --junitxml
  • Using --report or --output which are invalid
  • Missing the equals sign '='
3. Given this pytest command:
pytest tests/ --junitxml=report.xml
What will happen after running it?
medium
A. Tests run but no report file is created
B. Tests run and a file named report.xml is created with test results in JUnit XML format
C. Tests do not run; only the report.xml file is created
D. Tests run and results are printed only to console, no file created

Solution

  1. Step 1: Understand pytest command behavior

    The command runs tests in the tests/ folder and generates a JUnit XML report file named report.xml.
  2. Step 2: Check output expectations

    Since --junitxml=report.xml is specified, the file will be created with test results in XML format.
  3. Final Answer:

    Tests run and a file named report.xml is created with test results in JUnit XML format -> Option B
  4. Quick Check:

    --junitxml creates XML report file [OK]
Hint: --junitxml creates XML report file after tests run [OK]
Common Mistakes:
  • Thinking tests don't run when report option is used
  • Expecting report file without running tests
  • Assuming output only prints to console
4. You ran pytest --junitxml=results.xml but the file results.xml is empty. What is the most likely cause?
medium
A. No tests were collected or run
B. The --junitxml option is misspelled
C. The file path is invalid and pytest failed silently
D. JUnit XML reporting is not supported by pytest

Solution

  1. Step 1: Analyze empty report cause

    An empty JUnit XML file usually means no tests were found or executed.
  2. Step 2: Check other options

    The option --junitxml is correct, pytest supports it, and invalid paths usually cause errors, not silent empty files.
  3. Final Answer:

    No tests were collected or run -> Option A
  4. Quick Check:

    Empty XML = no tests run [OK]
Hint: Empty XML means no tests ran or were found [OK]
Common Mistakes:
  • Assuming option is misspelled without checking
  • Ignoring that no tests might be collected
  • Thinking pytest doesn't support JUnit XML
5. You want your CI system to fail the build on the first pytest test failure using --failfast and --maxfail=1 and also save a JUnit XML report named ci-report.xml. Which command should you use?
hard
A. pytest --junitxml=ci-report.xml --maxfail=1 --exitfirst
B. pytest --junitxml=ci-report.xml --exitfirst --maxfail=0
C. pytest --junitxml=ci-report.xml --failfast --maxfail=1
D. pytest --junitxml=ci-report.xml --failfast --maxfail=0

Solution

  1. Step 1: Understand options for failing fast

    To stop on first failure and fail the build, use --failfast and limit failures with --maxfail=1.
  2. Step 2: Confirm JUnit XML report option

    --junitxml=ci-report.xml saves the report file as required.
  3. Step 3: Evaluate options

    pytest --junitxml=ci-report.xml --failfast --maxfail=1 correctly combines --junitxml=ci-report.xml, --failfast, and --maxfail=1. Others have wrong or conflicting flags.
  4. Final Answer:

    pytest --junitxml=ci-report.xml --failfast --maxfail=1 -> Option C
  5. Quick Check:

    Fail fast + maxfail=1 + junitxml = pytest --junitxml=ci-report.xml --failfast --maxfail=1 [OK]
Hint: Use --failfast and --maxfail=1 with --junitxml for CI fail [OK]
Common Mistakes:
  • Confusing --exitfirst with --failfast
  • Setting maxfail=0 disables failure limit
  • Omitting --failfast causes tests to run fully