Running PyTest in Jenkins - Build an Automation Script

Run PyTest tests in Jenkins and verify test results

Preconditions (4)

Step 1: Open Jenkins dashboard

Step 2: Navigate to the Jenkins job for the project

Step 3: Configure the job to run a shell command: 'pytest --junitxml=results.xml'

Step 4: Add a post-build action to publish JUnit test results using 'results.xml'

Step 5: Save the job configuration

Step 6: Click 'Build Now' to run the job

Step 7: Wait for the build to complete

Step 8: Open the build details page

Step 9: Verify that the test results are displayed with passed and failed tests

✅ Expected Result: The Jenkins build runs PyTest tests, generates a JUnit XML report, and Jenkins displays the test results correctly showing which tests passed or failed.

Automation Requirements - pytest

Assertions Needed:

Verify the Jenkins build completes successfully

Verify the JUnit XML report file 'results.xml' is generated

Verify the test results contain expected passed and failed test counts

Best Practices:

Use explicit commands to run PyTest with JUnit XML output

Use Jenkins post-build actions to publish test reports

Keep test commands and report paths consistent

Use Jenkins pipeline or freestyle job with clear steps

Automated Solution

PyTest

import os
import subprocess
import xml.etree.ElementTree as ET
import unittest

class TestJenkinsPytestRun(unittest.TestCase):
    def test_pytest_run_and_report(self):
        # Run pytest with JUnit XML output
        result = subprocess.run(['pytest', '--junitxml=results.xml'], capture_output=True, text=True)
        # Assert pytest command ran successfully
        self.assertEqual(result.returncode, 0, f"Pytest failed: {result.stderr}")
        # Check if results.xml file exists
        self.assertTrue(os.path.exists('results.xml'), 'JUnit XML report not found')
        # Parse the XML report
        tree = ET.parse('results.xml')
        root = tree.getroot()
        # Extract test counts
        tests = int(root.attrib.get('tests', 0))
        failures = int(root.attrib.get('failures', 0))
        errors = int(root.attrib.get('errors', 0))
        skipped = int(root.attrib.get('skipped', 0))
        # Assert there is at least one test run
        self.assertGreater(tests, 0, 'No tests were run')
        # Assert no errors in test run
        self.assertEqual(errors, 0, 'There were errors in the test run')
        # Optionally assert failures count (can be zero or more)
        # Print summary
        print(f"Tests run: {tests}, Failures: {failures}, Skipped: {skipped}")

if __name__ == '__main__':
    unittest.main()

This script automates running PyTest with the --junitxml=results.xml option to generate a JUnit XML report file.

It uses Python's subprocess module to run the PyTest command and captures the output and return code to verify the test run succeeded.

Then it checks if the results.xml file was created, which Jenkins uses to display test results.

The script parses the XML report using xml.etree.ElementTree to extract the total tests, failures, errors, and skipped counts.

Assertions verify that tests were run, no errors occurred, and the report file exists.

This matches what Jenkins expects when configured to run PyTest and publish JUnit reports, ensuring the automation covers the manual test case steps.

Common Mistakes - 4 Pitfalls

{'mistake': 'Not generating the JUnit XML report with PyTest', 'why_bad': 'Jenkins cannot display test results without the XML report file.', 'correct_approach': "Always run PyTest with the '--junitxml=filename.xml' option to generate the report."}

Hardcoding file paths without considering Jenkins workspace

Ignoring PyTest return codes and errors

{'mistake': 'Not configuring Jenkins post-build action to publish test results', 'why_bad': "Test results won't be visible in Jenkins UI without publishing the report.", 'correct_approach': "Add 'Publish JUnit test result report' post-build action pointing to the XML report."}

Bonus Challenge

Now add data-driven testing with 3 different test input scenarios in PyTest and verify Jenkins runs all scenarios.

Show Hint