Parallel test execution in Jenkins - Commands & Configuration

Introduction

Running tests one after another can take a long time. Parallel test execution lets Jenkins run multiple tests at the same time, making testing faster and saving time.

When you have many tests and want to finish testing quickly before deployment.

When you want to use multiple machines or agents to run tests simultaneously.

When you want to reduce the waiting time for feedback after code changes.

When your tests are independent and can run without affecting each other.

When you want to speed up your continuous integration pipeline.

Config File - Jenkinsfile

Jenkinsfile

pipeline {
  agent any
  stages {
    stage('Parallel Tests') {
      parallel {
        'Test Suite 1': {
          steps {
            echo 'Running Test Suite 1'
            sh 'pytest tests/suite1'
          }
        },
        'Test Suite 2': {
          steps {
            echo 'Running Test Suite 2'
            sh 'pytest tests/suite2'
          }
        },
        'Test Suite 3': {
          steps {
            echo 'Running Test Suite 3'
            sh 'pytest tests/suite3'
          }
        }
      }
    }
  }
}

This Jenkinsfile defines a pipeline with one stage called 'Parallel Tests'. Inside this stage, three test suites run at the same time using the parallel block. Each test suite runs a separate pytest command. This setup speeds up testing by running tests simultaneously on any available agent.

Commands

This command updates the Jenkins job named 'my-pipeline' with the latest Jenkinsfile configuration to enable parallel test execution.

Terminal

jenkins-jobs --conf jenkins.ini update my-pipeline

Expected Output

Job 'my-pipeline' updated successfully.

This command triggers the build of the 'my-pipeline' job on Jenkins and waits for it to finish, showing the output live.

Terminal

java -jar jenkins-cli.jar -s http://localhost:8080 build my-pipeline -f

Expected Output

[Pipeline] Start of Pipeline

[Pipeline] stage

[Pipeline] { (Parallel Tests)

[Pipeline] parallel

[Pipeline] { (Test Suite 1)

Running Test Suite 1

=====================

...test output...

[Pipeline] }

[Pipeline] { (Test Suite 2)

Running Test Suite 2

=====================

...test output...

[Pipeline] }

[Pipeline] { (Test Suite 3)

Running Test Suite 3

=====================

...test output...

[Pipeline] }

[Pipeline] }

[Pipeline] End of Pipeline

Finished: SUCCESS

→

-f - Waits for the build to finish and streams the console output.

This command fetches the console output of the last build of 'my-pipeline' to verify the test results after parallel execution.

Terminal

java -jar jenkins-cli.jar -s http://localhost:8080 console my-pipeline

Expected Output

[Pipeline] Start of Pipeline

[Pipeline] stage

[Pipeline] { (Parallel Tests)

[Pipeline] parallel

[Pipeline] { (Test Suite 1)

Running Test Suite 1

=====================

...test output...

[Pipeline] }

[Pipeline] { (Test Suite 2)

Running Test Suite 2

=====================

...test output...

[Pipeline] }

[Pipeline] { (Test Suite 3)

Running Test Suite 3

=====================

...test output...

[Pipeline] }

[Pipeline] }

[Pipeline] End of Pipeline

Finished: SUCCESS

Key Concept

If you remember nothing else from this pattern, remember: use the Jenkins 'parallel' block inside a stage to run multiple test suites at the same time and speed up your pipeline.

Common Mistakes

Running tests sequentially without using the 'parallel' block.

This causes tests to run one after another, making the process slower and wasting resources.

Use the 'parallel' block in the Jenkinsfile to define multiple test stages that run simultaneously.

Running tests that depend on each other in parallel.

Tests may fail or produce incorrect results because they interfere with each other.

Ensure tests are independent and can safely run at the same time before using parallel execution.

Not waiting for the parallel tests to finish before moving to the next stage.

Subsequent stages may run before tests complete, causing errors or incomplete results.

Use Jenkins pipeline syntax properly so the pipeline waits for all parallel branches to finish before continuing.

Summary

Define multiple test stages inside a 'parallel' block in the Jenkinsfile to run tests simultaneously.

Trigger the Jenkins job and watch the tests run in parallel to save time.

Check the console output after the build to verify all tests passed successfully.