Running PyTest in Jenkins - Test Execution Trace

Test Overview

This test runs a simple PyTest test case inside Jenkins. It verifies that Jenkins can execute the PyTest command and that the test passes successfully.

Test Code - PyTest

PyTest

import pytest

def test_addition():
    assert 2 + 3 == 5

if __name__ == '__main__':
    pytest.main(['-v'])

Execution Trace - 5 Steps

Step	Action	System State	Assertion	Result
1	Jenkins job starts and runs the shell command 'pytest -v'	Jenkins agent environment is ready with Python and PyTest installed	—	PASS
2	PyTest discovers the test_addition function	PyTest test runner lists test_addition as a test to run	—	PASS
3	PyTest executes test_addition	Test function runs and performs assertion 2 + 3 == 5	Assert that 2 + 3 equals 5	PASS
4	PyTest reports test_addition PASSED	Test report shows 1 passed test	Verify test result is PASS	PASS
5	Jenkins captures PyTest output and marks build as SUCCESS	Jenkins console log shows PyTest output with PASS status	Jenkins build status is SUCCESS	PASS

Failure Scenario

Failing Condition: PyTest test_addition assertion fails or PyTest command is not found

Execution Trace Quiz - 3 Questions

What does Jenkins do first when running PyTest?

A Runs the shell command 'pytest -v'

B Directly marks the build as SUCCESS

C Skips test discovery

D Runs Python interpreter without PyTest

Key Result

Always verify that your test environment in Jenkins has the required tools installed and accessible before running tests to avoid command not found errors.

Practice

1. What is the main purpose of running pytest in Jenkins?

easy

A. To monitor server performance during tests

B. To deploy Python applications automatically

C. To write new Python test cases inside Jenkins

D. To automate running Python tests and see results in Jenkins

Solution

1. Step 1: Understand Jenkins role in testing

   Jenkins automates tasks like running tests without manual effort.

2. Step 2: Identify pytest's role

   Pytest runs Python tests and generates results for Jenkins to display.

3. Final Answer:

   To automate running Python tests and see results in Jenkins -> Option D

4. Quick Check:

   Automation of tests = A [OK]

Hint: Jenkins runs tests automatically to show results [OK]

Common Mistakes:

• Confusing test running with deployment
• Thinking Jenkins writes tests
• Assuming Jenkins monitors server performance

2. Which Jenkins Pipeline command correctly runs pytest and saves results in XML format?

easy

A. sh 'pytest --junitxml=results.xml'

B. sh 'pytest -o junit=results.xml'

C. sh 'pytest --xml=results.xml'

D. sh 'pytest --save=results.xml'

Solution

1. Step 1: Recall pytest XML output option

   The correct option to save results in XML is --junitxml=filename.

2. Step 2: Match Jenkins shell command syntax

   Jenkins uses sh to run shell commands, so sh 'pytest --junitxml=results.xml' is correct.

3. Final Answer:

   sh 'pytest --junitxml=results.xml' -> Option A

4. Quick Check:

   Correct pytest XML flag = D [OK]

Hint: Use --junitxml=filename to save pytest results [OK]

Common Mistakes:

• Using wrong pytest flags for XML output
• Incorrect Jenkins shell command syntax
• Confusing XML output with other formats

3. Given this Jenkins Pipeline snippet:

pipeline {
  agent any
  stages {
    stage('Test') {
      steps {
        sh 'pytest --junitxml=results.xml'
        junit 'results.xml'
      }
    }
  }
}

What will Jenkins display after running this pipeline?

medium

A. Test results summary with passed and failed tests

B. Only console output without test results

C. An error because junit step is missing

D. No output because tests are not run

Solution

1. Step 1: Understand the sh step

   The sh command runs pytest and generates results.xml with test results.

2. Step 2: Understand the junit step

   The junit step reads results.xml and shows test results in Jenkins UI.

3. Final Answer:

   Test results summary with passed and failed tests -> Option A

4. Quick Check:

   pytest + junit step = test summary shown [OK]

Hint: Use junit step to publish pytest XML results [OK]

Common Mistakes:

• Forgetting to add junit step to publish results
• Assuming pytest output alone shows results in Jenkins
• Confusing console logs with test reports

4. You run this Jenkins Pipeline:

sh 'pytest --junitxml=results.xml'
junit 'results.xml'

But Jenkins shows an error: FileNotFoundError: results.xml not found. What is the likely cause?

medium

A. Jenkins workspace is missing

B. Pytest did not run or failed before creating results.xml

C. The junit step is misspelled

D. The XML file is created but in a different directory

Solution

1. Step 1: Check if pytest ran successfully

   If pytest fails or does not run, it won't create results.xml.

2. Step 2: Confirm file existence before junit step

   The junit step needs the XML file; if missing, it errors out.

3. Final Answer:

   Pytest did not run or failed before creating results.xml -> Option B

4. Quick Check:

   Missing XML means pytest didn't create it [OK]

Hint: Check pytest success before junit step [OK]

Common Mistakes:

• Assuming junit step spelling causes file error
• Ignoring pytest failure logs
• Not verifying file path correctness

5. You want Jenkins to run pytest only on files changed in a Git branch and publish results. Which approach best fits this requirement?

hard

A. Manually run pytest locally and upload results to Jenkins

B. Run pytest on all files every time and ignore changed files

C. Use Jenkins Pipeline to detect changed files, run pytest on those, then publish with junit

D. Use Jenkins to deploy code without running tests

Solution

1. Step 1: Detect changed files in Jenkins Pipeline

   Use Git commands or plugins to find changed Python files in the branch.

2. Step 2: Run pytest only on those changed files and publish results

   Run pytest with paths of changed files, then use junit to show results.

3. Final Answer:

   Use Jenkins Pipeline to detect changed files, run pytest on those, then publish with junit -> Option C

4. Quick Check:

   Selective test run + publish = A [OK]

Hint: Detect changed files, run pytest on them, publish results [OK]

Common Mistakes:

• Running all tests every time wasting resources
• Skipping automated test runs
• Not publishing test results in Jenkins