Publish test results after pytest execution

Preconditions (2)

Step 1: Run pytest with the option to generate a JUnit XML report

Step 2: Verify that the JUnit XML report file is created in the specified location

Step 3: Open the generated XML report and check it contains the test results summary

✅ Expected Result: The JUnit XML report file is created successfully and contains accurate test results data

Automation Requirements - pytest

Assertions Needed:

Assert that the XML report file exists after test run

Assert that the XML report file contains the test case name and status

Best Practices:

Use pytest command line options to generate reports

Use Python's built-in libraries to verify report file existence and content

Keep test code clean and readable

Automated Solution

PyTest

import os
import xml.etree.ElementTree as ET
import subprocess

def test_run_and_publish_report():
    report_file = 'test_results.xml'
    # Run pytest with JUnit XML report generation
    result = subprocess.run(['pytest', 'test_sample.py', f'--junitxml={report_file}'], capture_output=True, text=True)
    
    # Assert pytest ran successfully
    assert result.returncode == 0, f"Pytest failed: {result.stderr}"
    
    # Assert report file exists
    assert os.path.exists(report_file), f"Report file {report_file} not found"
    
    # Parse XML report and check for test case name and status
    tree = ET.parse(report_file)
    root = tree.getroot()
    testcases = root.findall('.//testcase')
    assert len(testcases) > 0, "No test cases found in report"
    
    # Check that at least one test case has a 'name' attribute
    assert any(tc.attrib.get('name') for tc in testcases), "Test case names missing in report"
    
    # Check that no test case has a failure element (means all passed)
    failures = root.findall('.//failure')
    assert len(failures) == 0, "Some tests failed according to report"

This test function runs pytest on a sample test file with the --junitxml option to generate a test results report in XML format.

It uses subprocess.run to execute pytest and captures the output and return code to verify pytest ran successfully.

Then it checks if the XML report file was created using os.path.exists.

Next, it parses the XML report using xml.etree.ElementTree to find test case elements and asserts that test cases exist and have names.

Finally, it asserts that there are no failure elements in the report, meaning all tests passed.

This approach ensures the test results are published correctly and can be verified programmatically.

Common Mistakes - 3 Pitfalls

Not checking if the pytest command succeeded before verifying the report file

Hardcoding file paths without considering relative locations

Parsing the XML report without checking if the file exists

Bonus Challenge

Now add data-driven testing with 3 different test files, each generating its own report file

Show Hint