Running PyTest in Jenkins

Introduction

Running PyTest in Jenkins helps automate testing your Python code every time you make changes. This ensures your code works well and catches errors early.

You want to check your Python tests automatically after each code update.

You need to run tests on a remote server without manual intervention.

You want to see test results and reports in one place.

You want to prevent broken code from being added to your project.

You want to integrate testing into your software delivery process.

Syntax

PyTest

pipeline {
    agent any
    stages {
        stage('Run PyTest') {
            steps {
                sh 'pytest --junitxml=results.xml'
            }
        }
        stage('Publish Test Results') {
            steps {
                junit 'results.xml'
            }
        }
    }
}

This is a Jenkins Pipeline script written in Groovy.

The sh step runs the PyTest command in the shell.

Examples

Runs PyTest with default settings in a Jenkins shell step.

PyTest

sh 'pytest'

Runs PyTest and saves test results in XML format for Jenkins to read.

PyTest

sh 'pytest --junitxml=results.xml'

Publishes the test results in Jenkins using the XML file generated by PyTest.

PyTest

junit 'results.xml'

Sample Program

This Jenkins Pipeline runs PyTest to execute tests and then publishes the test results in Jenkins.

PyTest

pipeline {
    agent any
    stages {
        stage('Run PyTest') {
            steps {
                sh 'pytest --junitxml=results.xml'
            }
        }
        stage('Publish Test Results') {
            steps {
                junit 'results.xml'
            }
        }
    }
}

Important Notes

Make sure Python and PyTest are installed on the Jenkins agent machine.

The --junitxml option creates a file Jenkins can read to show test results.

Use the Jenkins junit step to display test results and mark build status.

Summary

Running PyTest in Jenkins automates your Python testing process.

Use a Jenkins Pipeline with sh 'pytest --junitxml=results.xml' to run tests.

Publish results with the junit step to see test outcomes in Jenkins.

Practice

1. What is the main purpose of running pytest in Jenkins?

easy

A. To monitor server performance during tests

B. To deploy Python applications automatically

C. To write new Python test cases inside Jenkins

D. To automate running Python tests and see results in Jenkins

Solution

1. Step 1: Understand Jenkins role in testing

   Jenkins automates tasks like running tests without manual effort.

2. Step 2: Identify pytest's role

   Pytest runs Python tests and generates results for Jenkins to display.

3. Final Answer:

   To automate running Python tests and see results in Jenkins -> Option D

4. Quick Check:

   Automation of tests = A [OK]

Hint: Jenkins runs tests automatically to show results [OK]

Common Mistakes:

• Confusing test running with deployment
• Thinking Jenkins writes tests
• Assuming Jenkins monitors server performance

2. Which Jenkins Pipeline command correctly runs pytest and saves results in XML format?

easy

A. sh 'pytest --junitxml=results.xml'

B. sh 'pytest -o junit=results.xml'

C. sh 'pytest --xml=results.xml'

D. sh 'pytest --save=results.xml'

Solution

1. Step 1: Recall pytest XML output option

   The correct option to save results in XML is --junitxml=filename.

2. Step 2: Match Jenkins shell command syntax

   Jenkins uses sh to run shell commands, so sh 'pytest --junitxml=results.xml' is correct.

3. Final Answer:

   sh 'pytest --junitxml=results.xml' -> Option A

4. Quick Check:

   Correct pytest XML flag = D [OK]

Hint: Use --junitxml=filename to save pytest results [OK]

Common Mistakes:

• Using wrong pytest flags for XML output
• Incorrect Jenkins shell command syntax
• Confusing XML output with other formats

3. Given this Jenkins Pipeline snippet:

pipeline {
  agent any
  stages {
    stage('Test') {
      steps {
        sh 'pytest --junitxml=results.xml'
        junit 'results.xml'
      }
    }
  }
}

What will Jenkins display after running this pipeline?

medium

A. Test results summary with passed and failed tests

B. Only console output without test results

C. An error because junit step is missing

D. No output because tests are not run

Solution

1. Step 1: Understand the sh step

   The sh command runs pytest and generates results.xml with test results.

2. Step 2: Understand the junit step

   The junit step reads results.xml and shows test results in Jenkins UI.

3. Final Answer:

   Test results summary with passed and failed tests -> Option A

4. Quick Check:

   pytest + junit step = test summary shown [OK]

Hint: Use junit step to publish pytest XML results [OK]

Common Mistakes:

• Forgetting to add junit step to publish results
• Assuming pytest output alone shows results in Jenkins
• Confusing console logs with test reports

4. You run this Jenkins Pipeline:

sh 'pytest --junitxml=results.xml'
junit 'results.xml'

But Jenkins shows an error: FileNotFoundError: results.xml not found. What is the likely cause?

medium

A. Jenkins workspace is missing

B. Pytest did not run or failed before creating results.xml

C. The junit step is misspelled

D. The XML file is created but in a different directory

Solution

1. Step 1: Check if pytest ran successfully

   If pytest fails or does not run, it won't create results.xml.

2. Step 2: Confirm file existence before junit step

   The junit step needs the XML file; if missing, it errors out.

3. Final Answer:

   Pytest did not run or failed before creating results.xml -> Option B

4. Quick Check:

   Missing XML means pytest didn't create it [OK]

Hint: Check pytest success before junit step [OK]

Common Mistakes:

• Assuming junit step spelling causes file error
• Ignoring pytest failure logs
• Not verifying file path correctness

5. You want Jenkins to run pytest only on files changed in a Git branch and publish results. Which approach best fits this requirement?

hard

A. Manually run pytest locally and upload results to Jenkins

B. Run pytest on all files every time and ignore changed files

C. Use Jenkins Pipeline to detect changed files, run pytest on those, then publish with junit

D. Use Jenkins to deploy code without running tests

Solution

1. Step 1: Detect changed files in Jenkins Pipeline

   Use Git commands or plugins to find changed Python files in the branch.

2. Step 2: Run pytest only on those changed files and publish results

   Run pytest with paths of changed files, then use junit to show results.

3. Final Answer:

   Use Jenkins Pipeline to detect changed files, run pytest on those, then publish with junit -> Option C

4. Quick Check:

   Selective test run + publish = A [OK]

Hint: Detect changed files, run pytest on them, publish results [OK]

Common Mistakes:

• Running all tests every time wasting resources
• Skipping automated test runs
• Not publishing test results in Jenkins