Running PyTest in Jenkins - Framework Patterns

Framework Mode - Running PyTest in Jenkins

Folder Structure for PyTest Project

project-root/
├── tests/
│   ├── test_example.py
│   └── conftest.py
├── src/
│   └── application_code.py
├── pytest.ini
├── requirements.txt
└── Jenkinsfile
  

Test Framework Layers

• Tests: Located in tests/ folder, contains test cases using PyTest syntax.
• Fixtures: Defined in conftest.py for setup and teardown reusable code.
• Application Code: In src/ folder, the code under test.
• Configuration: pytest.ini for PyTest settings and markers.
• CI Pipeline: Jenkinsfile defines Jenkins build and test steps.

Configuration Patterns

• pytest.ini: Configure test markers, addopts (e.g., --junitxml=reports/result.xml), and test paths.
• Environment Variables: Use Jenkins environment variables to pass credentials or environment info securely.
• Jenkinsfile: Define stages for installing dependencies, running tests, and archiving reports.

  pipeline {
    agent any
    stages {
      stage('Install') {
        steps {
          sh 'python -m venv venv'
          sh './venv/bin/pip install -r requirements.txt'
        }
      }
      stage('Test') {
        steps {
          sh './venv/bin/pytest --junitxml=reports/result.xml'
        }
      }
    }
    post {
      always {
        junit 'reports/result.xml'
        archiveArtifacts artifacts: 'reports/**', allowEmptyArchive: true
      }
    }
  }

Test Reporting and CI/CD Integration

• JUnit XML Reports: PyTest generates result.xml for Jenkins to parse test results.
• Jenkins JUnit Plugin: Reads XML reports to show pass/fail and test details in Jenkins UI.
• Archiving Artifacts: Save logs and reports for later review.
• Pipeline Feedback: Jenkins marks build as failed if tests fail, enabling quick feedback.

Best Practices for Running PyTest in Jenkins

• Use virtual environments to isolate dependencies in Jenkins builds.
• Generate machine-readable test reports (JUnit XML) for Jenkins integration.
• Keep test data and credentials out of code; use Jenkins credentials and environment variables.
• Structure Jenkinsfile with clear stages: setup, test, report.
• Archive test reports and logs for debugging failed builds.

Self Check

Where in this framework structure would you add a new fixture to set up a database connection for tests?

Key Result

Organize PyTest tests and configuration for Jenkins CI with clear folder structure, JUnit reporting, and pipeline stages.

Practice

1. What is the main purpose of running pytest in Jenkins?

easy

A. To monitor server performance during tests

B. To deploy Python applications automatically

C. To write new Python test cases inside Jenkins

D. To automate running Python tests and see results in Jenkins

Solution

1. Step 1: Understand Jenkins role in testing

   Jenkins automates tasks like running tests without manual effort.

2. Step 2: Identify pytest's role

   Pytest runs Python tests and generates results for Jenkins to display.

3. Final Answer:

   To automate running Python tests and see results in Jenkins -> Option D

4. Quick Check:

   Automation of tests = A [OK]

Hint: Jenkins runs tests automatically to show results [OK]

Common Mistakes:

• Confusing test running with deployment
• Thinking Jenkins writes tests
• Assuming Jenkins monitors server performance

2. Which Jenkins Pipeline command correctly runs pytest and saves results in XML format?

easy

A. sh 'pytest --junitxml=results.xml'

B. sh 'pytest -o junit=results.xml'

C. sh 'pytest --xml=results.xml'

D. sh 'pytest --save=results.xml'

Solution

1. Step 1: Recall pytest XML output option

   The correct option to save results in XML is --junitxml=filename.

2. Step 2: Match Jenkins shell command syntax

   Jenkins uses sh to run shell commands, so sh 'pytest --junitxml=results.xml' is correct.

3. Final Answer:

   sh 'pytest --junitxml=results.xml' -> Option A

4. Quick Check:

   Correct pytest XML flag = D [OK]

Hint: Use --junitxml=filename to save pytest results [OK]

Common Mistakes:

• Using wrong pytest flags for XML output
• Incorrect Jenkins shell command syntax
• Confusing XML output with other formats

3. Given this Jenkins Pipeline snippet:

pipeline {
  agent any
  stages {
    stage('Test') {
      steps {
        sh 'pytest --junitxml=results.xml'
        junit 'results.xml'
      }
    }
  }
}

What will Jenkins display after running this pipeline?

medium

A. Test results summary with passed and failed tests

B. Only console output without test results

C. An error because junit step is missing

D. No output because tests are not run

Solution

1. Step 1: Understand the sh step

   The sh command runs pytest and generates results.xml with test results.

2. Step 2: Understand the junit step

   The junit step reads results.xml and shows test results in Jenkins UI.

3. Final Answer:

   Test results summary with passed and failed tests -> Option A

4. Quick Check:

   pytest + junit step = test summary shown [OK]

Hint: Use junit step to publish pytest XML results [OK]

Common Mistakes:

• Forgetting to add junit step to publish results
• Assuming pytest output alone shows results in Jenkins
• Confusing console logs with test reports

4. You run this Jenkins Pipeline:

sh 'pytest --junitxml=results.xml'
junit 'results.xml'

But Jenkins shows an error: FileNotFoundError: results.xml not found. What is the likely cause?

medium

A. Jenkins workspace is missing

B. Pytest did not run or failed before creating results.xml

C. The junit step is misspelled

D. The XML file is created but in a different directory

Solution

1. Step 1: Check if pytest ran successfully

   If pytest fails or does not run, it won't create results.xml.

2. Step 2: Confirm file existence before junit step

   The junit step needs the XML file; if missing, it errors out.

3. Final Answer:

   Pytest did not run or failed before creating results.xml -> Option B

4. Quick Check:

   Missing XML means pytest didn't create it [OK]

Hint: Check pytest success before junit step [OK]

Common Mistakes:

• Assuming junit step spelling causes file error
• Ignoring pytest failure logs
• Not verifying file path correctness

5. You want Jenkins to run pytest only on files changed in a Git branch and publish results. Which approach best fits this requirement?

hard

A. Manually run pytest locally and upload results to Jenkins

B. Run pytest on all files every time and ignore changed files

C. Use Jenkins Pipeline to detect changed files, run pytest on those, then publish with junit

D. Use Jenkins to deploy code without running tests

Solution

1. Step 1: Detect changed files in Jenkins Pipeline

   Use Git commands or plugins to find changed Python files in the branch.

2. Step 2: Run pytest only on those changed files and publish results

   Run pytest with paths of changed files, then use junit to show results.

3. Final Answer:

   Use Jenkins Pipeline to detect changed files, run pytest on those, then publish with junit -> Option C

4. Quick Check:

   Selective test run + publish = A [OK]

Hint: Detect changed files, run pytest on them, publish results [OK]

Common Mistakes:

• Running all tests every time wasting resources
• Skipping automated test runs
• Not publishing test results in Jenkins