Running PyTest in Jenkins - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is Jenkins used for in running PyTest?

Jenkins automates running PyTest tests regularly or on code changes, helping catch errors early.

beginner

Which Jenkins plugin helps run shell commands like PyTest?

The 'Execute shell' build step lets Jenkins run PyTest commands in a job.

beginner

How do you specify the PyTest command in Jenkins to run tests in the 'tests' folder?

Use the command: pytest tests/ in the Jenkins build step.

intermediate

Why add a 'Post-build Action' to publish PyTest results in Jenkins?

It shows test results and trends in Jenkins UI, making it easy to see test success or failure.

intermediate

What file format does PyTest use to output test results for Jenkins to read?

PyTest can output results in JUnit XML format using --junitxml=report.xml.

What is the first step to run PyTest in Jenkins?

A Create a Jenkins job and add a build step to run PyTest

B Install PyTest plugin in Jenkins

C Write tests in Jenkinsfile

D Restart Jenkins server

You start by creating a Jenkins job and adding a build step to run PyTest commands.

Which command runs PyTest tests in a folder named 'tests'?

A pytest -run tests

B pytest run tests

C run pytest tests

D pytest tests/

The correct command is 'pytest tests/' to run all tests inside the 'tests' folder.

How can Jenkins display PyTest test results in its interface?

A By running PyTest in verbose mode

B By publishing JUnit XML reports generated by PyTest

C By installing PyTest UI plugin

D By saving test logs as text files

Jenkins reads JUnit XML reports to show test results and trends.

What option tells PyTest to create a JUnit XML report?

A --junitxml=report.xml

B --xmlreport=report.xml

C --report=xml

D --output=report.xml

The option '--junitxml=report.xml' generates a JUnit XML report file.

Which Jenkins build step is commonly used to run PyTest commands?

A Execute Python script

B Invoke Ant

C Execute shell

D Run batch command

The 'Execute shell' step runs shell commands like PyTest in Jenkins jobs.

Explain how to set up Jenkins to run PyTest tests automatically.

Think about job creation, running tests, reporting, and viewing results.

Describe why generating a JUnit XML report is important when running PyTest in Jenkins.

Focus on how Jenkins uses the report to help teams.

Practice

1. What is the main purpose of running pytest in Jenkins?

easy

A. To monitor server performance during tests

B. To deploy Python applications automatically

C. To write new Python test cases inside Jenkins

D. To automate running Python tests and see results in Jenkins

Solution

1. Step 1: Understand Jenkins role in testing

   Jenkins automates tasks like running tests without manual effort.

2. Step 2: Identify pytest's role

   Pytest runs Python tests and generates results for Jenkins to display.

3. Final Answer:

   To automate running Python tests and see results in Jenkins -> Option D

4. Quick Check:

   Automation of tests = A [OK]

Hint: Jenkins runs tests automatically to show results [OK]

Common Mistakes:

• Confusing test running with deployment
• Thinking Jenkins writes tests
• Assuming Jenkins monitors server performance

2. Which Jenkins Pipeline command correctly runs pytest and saves results in XML format?

easy

A. sh 'pytest --junitxml=results.xml'

B. sh 'pytest -o junit=results.xml'

C. sh 'pytest --xml=results.xml'

D. sh 'pytest --save=results.xml'

Solution

1. Step 1: Recall pytest XML output option

   The correct option to save results in XML is --junitxml=filename.

2. Step 2: Match Jenkins shell command syntax

   Jenkins uses sh to run shell commands, so sh 'pytest --junitxml=results.xml' is correct.

3. Final Answer:

   sh 'pytest --junitxml=results.xml' -> Option A

4. Quick Check:

   Correct pytest XML flag = D [OK]

Hint: Use --junitxml=filename to save pytest results [OK]

Common Mistakes:

• Using wrong pytest flags for XML output
• Incorrect Jenkins shell command syntax
• Confusing XML output with other formats

3. Given this Jenkins Pipeline snippet:

pipeline {
  agent any
  stages {
    stage('Test') {
      steps {
        sh 'pytest --junitxml=results.xml'
        junit 'results.xml'
      }
    }
  }
}

What will Jenkins display after running this pipeline?

medium

A. Test results summary with passed and failed tests

B. Only console output without test results

C. An error because junit step is missing

D. No output because tests are not run

Solution

1. Step 1: Understand the sh step

   The sh command runs pytest and generates results.xml with test results.

2. Step 2: Understand the junit step

   The junit step reads results.xml and shows test results in Jenkins UI.

3. Final Answer:

   Test results summary with passed and failed tests -> Option A

4. Quick Check:

   pytest + junit step = test summary shown [OK]

Hint: Use junit step to publish pytest XML results [OK]

Common Mistakes:

• Forgetting to add junit step to publish results
• Assuming pytest output alone shows results in Jenkins
• Confusing console logs with test reports

4. You run this Jenkins Pipeline:

sh 'pytest --junitxml=results.xml'
junit 'results.xml'

But Jenkins shows an error: FileNotFoundError: results.xml not found. What is the likely cause?

medium

A. Jenkins workspace is missing

B. Pytest did not run or failed before creating results.xml

C. The junit step is misspelled

D. The XML file is created but in a different directory

Solution

1. Step 1: Check if pytest ran successfully

   If pytest fails or does not run, it won't create results.xml.

2. Step 2: Confirm file existence before junit step

   The junit step needs the XML file; if missing, it errors out.

3. Final Answer:

   Pytest did not run or failed before creating results.xml -> Option B

4. Quick Check:

   Missing XML means pytest didn't create it [OK]

Hint: Check pytest success before junit step [OK]

Common Mistakes:

• Assuming junit step spelling causes file error
• Ignoring pytest failure logs
• Not verifying file path correctness

5. You want Jenkins to run pytest only on files changed in a Git branch and publish results. Which approach best fits this requirement?

hard

A. Manually run pytest locally and upload results to Jenkins

B. Run pytest on all files every time and ignore changed files

C. Use Jenkins Pipeline to detect changed files, run pytest on those, then publish with junit

D. Use Jenkins to deploy code without running tests

Solution

1. Step 1: Detect changed files in Jenkins Pipeline

   Use Git commands or plugins to find changed Python files in the branch.

2. Step 2: Run pytest only on those changed files and publish results

   Run pytest with paths of changed files, then use junit to show results.

3. Final Answer:

   Use Jenkins Pipeline to detect changed files, run pytest on those, then publish with junit -> Option C

4. Quick Check:

   Selective test run + publish = A [OK]

Hint: Detect changed files, run pytest on them, publish results [OK]

Common Mistakes:

• Running all tests every time wasting resources
• Skipping automated test runs
• Not publishing test results in Jenkins