Test result publishing in PyTest - Build an Automation Script

Publish test results after pytest execution

Preconditions (2)

Step 1: Run pytest with the option to generate a JUnit XML report

Step 2: Verify that the JUnit XML report file is created in the specified location

Step 3: Open the generated XML report and check it contains the test results summary

✅ Expected Result: The JUnit XML report file is created successfully and contains accurate test results data

Automation Requirements - pytest

Assertions Needed:

Assert that the XML report file exists after test run

Assert that the XML report file contains the test case name and status

Best Practices:

Use pytest command line options to generate reports

Use Python's built-in libraries to verify report file existence and content

Keep test code clean and readable

Automated Solution

PyTest

import os
import xml.etree.ElementTree as ET
import subprocess

def test_run_and_publish_report():
    report_file = 'test_results.xml'
    # Run pytest with JUnit XML report generation
    result = subprocess.run(['pytest', 'test_sample.py', f'--junitxml={report_file}'], capture_output=True, text=True)
    
    # Assert pytest ran successfully
    assert result.returncode == 0, f"Pytest failed: {result.stderr}"
    
    # Assert report file exists
    assert os.path.exists(report_file), f"Report file {report_file} not found"
    
    # Parse XML report and check for test case name and status
    tree = ET.parse(report_file)
    root = tree.getroot()
    testcases = root.findall('.//testcase')
    assert len(testcases) > 0, "No test cases found in report"
    
    # Check that at least one test case has a 'name' attribute
    assert any(tc.attrib.get('name') for tc in testcases), "Test case names missing in report"
    
    # Check that no test case has a failure element (means all passed)
    failures = root.findall('.//failure')
    assert len(failures) == 0, "Some tests failed according to report"

This test function runs pytest on a sample test file with the --junitxml option to generate a test results report in XML format.

It uses subprocess.run to execute pytest and captures the output and return code to verify pytest ran successfully.

Then it checks if the XML report file was created using os.path.exists.

Next, it parses the XML report using xml.etree.ElementTree to find test case elements and asserts that test cases exist and have names.

Finally, it asserts that there are no failure elements in the report, meaning all tests passed.

This approach ensures the test results are published correctly and can be verified programmatically.

Common Mistakes - 3 Pitfalls

Not checking if the pytest command succeeded before verifying the report file

Hardcoding file paths without considering relative locations

Parsing the XML report without checking if the file exists

Bonus Challenge

Now add data-driven testing with 3 different test files, each generating its own report file

Show Hint

Practice

1. What is the main purpose of publishing test results in pytest?

easy

A. To change the test code behavior

B. To speed up test execution time

C. To write more test cases automatically

D. To share test outcomes with team members or tools

Solution

1. Step 1: Understand test result publishing

   Publishing test results means sharing what happened during tests with others.

2. Step 2: Identify the purpose in pytest context

   In pytest, publishing results helps teams see which tests passed or failed.

3. Final Answer:

   To share test outcomes with team members or tools -> Option D

4. Quick Check:

   Test result publishing = sharing outcomes [OK]

Hint: Publishing means sharing test results with others [OK]

Common Mistakes:

• Thinking publishing speeds up tests
• Confusing publishing with writing tests
• Believing publishing changes test code

2. Which pytest command correctly saves test results to an XML file named results.xml?

easy

A. pytest --junitxml=results.xml

B. pytest --savexml=results.xml

C. pytest --output=results.xml

D. pytest --xmlfile=results.xml

Solution

1. Step 1: Recall pytest command for XML report

   The correct option uses --junitxml to save results in XML format.

2. Step 2: Match the command with the filename

   Using pytest --junitxml=results.xml saves the test report as results.xml.

3. Final Answer:

   pytest --junitxml=results.xml -> Option A

4. Quick Check:

   Use --junitxml to save XML report [OK]

Hint: Use --junitxml=filename.xml to save results [OK]

Common Mistakes:

• Using --savexml instead of --junitxml
• Using --output or --xmlfile which are invalid
• Forgetting the equals sign '='

3. Given this pytest command:

pytest --junitxml=report.xml

What will happen after running tests?

medium

A. Test results will be saved in a file named report.xml

B. Tests will not run due to syntax error

C. Tests will run but no results will be saved

D. Test results will be printed only on the console

Solution

1. Step 1: Understand the command option

   The option --junitxml=report.xml tells pytest to save results in XML format to the file report.xml.

2. Step 2: Predict the outcome after running tests

   Tests run normally and results are saved in the specified XML file.

3. Final Answer:

   Test results will be saved in a file named report.xml -> Option A

4. Quick Check:

   --junitxml saves results to XML file [OK]

Hint: Look for --junitxml to save results to XML file [OK]

Common Mistakes:

• Thinking results only print on console
• Assuming tests won't run with this option
• Confusing file saving with syntax errors

4. You run pytest --junitxml report.xml but no report.xml file is created. What is the likely problem?

medium

A. pytest does not support XML reports

B. The command is missing an equals sign between option and filename

C. The test files are empty so no report is generated

D. The filename must be results.xml exactly

Solution

1. Step 1: Check the command syntax

   The correct syntax requires an equals sign: --junitxml=report.xml.

2. Step 2: Understand effect of missing equals sign

   Without '=', pytest treats report.xml as a positional argument, so no file is created.

3. Final Answer:

   The command is missing an equals sign between option and filename -> Option B

4. Quick Check:

   Use '=' after --junitxml to save file [OK]

Hint: Remember to use '=' after --junitxml option [OK]

Common Mistakes:

• Omitting the equals sign in command
• Thinking empty tests prevent report creation
• Believing pytest can't create XML reports

5. You want to publish pytest results in XML and also ensure the report includes test durations. Which command should you use?

hard

A. pytest --xmlreport=report.xml --showdurations

B. pytest --junitxml=report.xml --duration=all

C. pytest --junitxml=report.xml --durations=0

D. pytest --junitxml report.xml --durations

Solution

1. Step 1: Identify correct option for XML report

   The option --junitxml=report.xml correctly saves results in XML format.

2. Step 2: Add option to show all test durations

   The option --durations=0 tells pytest to show durations for all tests.

3. Step 3: Verify combined command correctness

   Combining both options as pytest --junitxml=report.xml --durations=0 is valid and achieves the goal.

4. Final Answer:

   pytest --junitxml=report.xml --durations=0 -> Option C

5. Quick Check:

   Use --junitxml and --durations=0 together [OK]

Hint: Use --junitxml=filename and --durations=0 for full report [OK]

Common Mistakes:

• Using wrong option names like --xmlreport
• Missing equals sign in options
• Using incorrect values like --duration=all