Parallel test execution in Jenkins - Commands & Configuration

Introduction

Running tests one after another can take a long time. Parallel test execution lets Jenkins run multiple tests at the same time, making testing faster and saving time.

When you have many tests and want to finish testing quickly before deployment.

When you want to use multiple machines or agents to run tests simultaneously.

When you want to reduce the waiting time for feedback after code changes.

When your tests are independent and can run without affecting each other.

When you want to speed up your continuous integration pipeline.

Config File - Jenkinsfile

Jenkinsfile

pipeline {
  agent any
  stages {
    stage('Parallel Tests') {
      parallel {
        'Test Suite 1': {
          steps {
            echo 'Running Test Suite 1'
            sh 'pytest tests/suite1'
          }
        },
        'Test Suite 2': {
          steps {
            echo 'Running Test Suite 2'
            sh 'pytest tests/suite2'
          }
        },
        'Test Suite 3': {
          steps {
            echo 'Running Test Suite 3'
            sh 'pytest tests/suite3'
          }
        }
      }
    }
  }
}

This Jenkinsfile defines a pipeline with one stage called 'Parallel Tests'. Inside this stage, three test suites run at the same time using the parallel block. Each test suite runs a separate pytest command. This setup speeds up testing by running tests simultaneously on any available agent.

Commands

This command updates the Jenkins job named 'my-pipeline' with the latest Jenkinsfile configuration to enable parallel test execution.

Terminal

jenkins-jobs --conf jenkins.ini update my-pipeline

Expected Output

Job 'my-pipeline' updated successfully.

This command triggers the build of the 'my-pipeline' job on Jenkins and waits for it to finish, showing the output live.

Terminal

java -jar jenkins-cli.jar -s http://localhost:8080 build my-pipeline -f

Expected Output

[Pipeline] Start of Pipeline

[Pipeline] stage

[Pipeline] { (Parallel Tests)

[Pipeline] parallel

[Pipeline] { (Test Suite 1)

Running Test Suite 1

=====================

...test output...

[Pipeline] }

[Pipeline] { (Test Suite 2)

Running Test Suite 2

=====================

...test output...

[Pipeline] }

[Pipeline] { (Test Suite 3)

Running Test Suite 3

=====================

...test output...

[Pipeline] }

[Pipeline] }

[Pipeline] End of Pipeline

Finished: SUCCESS

→

-f - Waits for the build to finish and streams the console output.

This command fetches the console output of the last build of 'my-pipeline' to verify the test results after parallel execution.

Terminal

java -jar jenkins-cli.jar -s http://localhost:8080 console my-pipeline

Expected Output

[Pipeline] Start of Pipeline

[Pipeline] stage

[Pipeline] { (Parallel Tests)

[Pipeline] parallel

[Pipeline] { (Test Suite 1)

Running Test Suite 1

=====================

...test output...

[Pipeline] }

[Pipeline] { (Test Suite 2)

Running Test Suite 2

=====================

...test output...

[Pipeline] }

[Pipeline] { (Test Suite 3)

Running Test Suite 3

=====================

...test output...

[Pipeline] }

[Pipeline] }

[Pipeline] End of Pipeline

Finished: SUCCESS

Key Concept

If you remember nothing else from this pattern, remember: use the Jenkins 'parallel' block inside a stage to run multiple test suites at the same time and speed up your pipeline.

Common Mistakes

Running tests sequentially without using the 'parallel' block.

This causes tests to run one after another, making the process slower and wasting resources.

Use the 'parallel' block in the Jenkinsfile to define multiple test stages that run simultaneously.

Running tests that depend on each other in parallel.

Tests may fail or produce incorrect results because they interfere with each other.

Ensure tests are independent and can safely run at the same time before using parallel execution.

Not waiting for the parallel tests to finish before moving to the next stage.

Subsequent stages may run before tests complete, causing errors or incomplete results.

Use Jenkins pipeline syntax properly so the pipeline waits for all parallel branches to finish before continuing.

Summary

Define multiple test stages inside a 'parallel' block in the Jenkinsfile to run tests simultaneously.

Trigger the Jenkins job and watch the tests run in parallel to save time.

Check the console output after the build to verify all tests passed successfully.

Practice

1. What is the main purpose of using the parallel block in a Jenkins pipeline?

easy

A. To stop the pipeline if any test fails

B. To run multiple test groups at the same time to save time

C. To run tests one after another in a sequence

D. To deploy the application automatically after tests

Solution

1. Step 1: Understand the parallel block function

   The parallel block in Jenkins allows running multiple tasks simultaneously.

2. Step 2: Identify the benefit of parallel execution

   Running tests in parallel saves time by executing multiple test groups at once instead of sequentially.

3. Final Answer:

   To run multiple test groups at the same time to save time -> Option B

4. Quick Check:

   Parallel execution = run tests simultaneously [OK]

Hint: Parallel means running tasks at the same time [OK]

Common Mistakes:

• Thinking parallel runs tests one by one
• Confusing parallel with deployment steps
• Assuming parallel stops on first failure

2. Which of the following is the correct syntax to run two test stages in parallel in a Jenkins declarative pipeline?

easy

A. parallel {
stage('Test1') { steps { echo 'Run Test1' } }
stage('Test2') { steps { echo 'Run Test2' } }
}

B. parallel(
Test1: { stage('Test1') { steps { echo 'Run Test1' } } },
Test2: { stage('Test2') { steps { echo 'Run Test2' } } }
)

C. parallel(
Test1: { echo 'Run Test1' },
Test2: { echo 'Run Test2' }
)

D. parallel(
Test1: { steps { echo 'Run Test1' } },
Test2: { steps { echo 'Run Test2' } }
)

Solution

1. Step 1: Recall Jenkins parallel syntax

   In declarative pipelines, parallel takes a map of closures using scripted syntax with direct steps like echo.

2. Step 2: Check which option uses correct closure with direct steps

   parallel( Test1: { echo 'Run Test1' }, Test2: { echo 'Run Test2' } ) correctly uses parallel with keys and closures containing direct echo.

3. Final Answer:

   parallel( Test1: { echo 'Run Test1' }, Test2: { echo 'Run Test2' } ) -> Option C

4. Quick Check:

   parallel map with direct echo closures = parallel( Test1: { echo 'Run Test1' }, Test2: { echo 'Run Test2' } ) [OK]

Hint: parallel needs map keys with closures containing direct steps like echo [OK]

Common Mistakes:

• Using stage inside parallel directly (wrong in scripted)
• Adding steps block inside closures (declarative syntax mismatch)
• Using braces incorrectly for parallel syntax

3. Given the following Jenkins scripted pipeline snippet, what will be the output?

parallel(
  TestA: {
    echo 'Running Test A'
  },
  TestB: {
    echo 'Running Test B'
  }
)
echo 'All tests done'

medium

A. Running Test A\nRunning Test B\nAll tests done

B. Running Test A\nAll tests done\nRunning Test B

C. All tests done\nRunning Test A\nRunning Test B

D. Running Test B\nRunning Test A\nAll tests done

Solution

1. Step 1: Understand parallel execution output order

   Both TestA and TestB run at the same time, so their echo outputs appear before the next command.

2. Step 2: Identify the sequence of echo commands

   After both parallel tasks finish, the final echo 'All tests done' runs.

3. Final Answer:

   Running Test A\nRunning Test B\nAll tests done -> Option A

4. Quick Check:

   Parallel tests output first, then final echo [OK]

Hint: Parallel echoes appear before next sequential echo [OK]

Common Mistakes:

• Assuming output order is strictly sequential
• Thinking 'All tests done' prints before tests
• Confusing parallel output interleaving

4. You wrote this Jenkins pipeline snippet to run tests in parallel but it fails with a syntax error:

parallel(
  Test1: {
    stage('Test1') {
      steps {
        echo 'Running Test1'
      }
    }
  },
  Test2: {
    stage('Test2') {
      steps {
        echo 'Running Test2'
      }
    }
  }
)

What is the main cause of the error?

medium

A. Using parallel block outside a node block

B. Missing commas between parallel map entries

C. Incorrect indentation of the echo commands

D. Using stage and steps inside scripted pipeline closures

Solution

1. Step 1: Identify pipeline type and syntax rules

   In scripted pipelines, stage and steps blocks are not used inside closures passed to parallel.

2. Step 2: Recognize correct scripted pipeline parallel syntax

   Scripted pipelines use simple closures with commands like echo directly, without stage or steps.

3. Final Answer:

   Using stage and steps inside scripted pipeline closures -> Option D

4. Quick Check:

   Scripted parallel closures exclude stage/steps blocks [OK]

Hint: Scripted parallel closures don't use stage or steps blocks [OK]

Common Mistakes:

• Mixing declarative syntax inside scripted pipeline
• Forgetting commas between map entries (not the main error here)
• Assuming indentation causes syntax errors

5. You want to run three test suites in parallel in a Jenkins declarative pipeline and fail the entire build if any test fails. Which snippet correctly achieves this?

hard

A. stages {
stage('Parallel Tests') {
parallel {
Test1: { steps { sh 'run_test1.sh' } }
Test2: { steps { sh 'run_test2.sh' } }
Test3: { steps { sh 'run_test3.sh' } }
}
}
}

B. parallel {
stage('Test1') { steps { sh 'run_test1.sh' } }
stage('Test2') { steps { sh 'run_test2.sh' } }
stage('Test3') { steps { sh 'run_test3.sh' } }
}

C. stages {
parallel {
stage('Test1') { steps { sh 'run_test1.sh' } }
stage('Test2') { steps { sh 'run_test2.sh' } }
stage('Test3') { steps { sh 'run_test3.sh' } }
}
}

D. stage('Parallel Tests') {
parallel(
Test1: { sh 'run_test1.sh' },
Test2: { sh 'run_test2.sh' },
Test3: { sh 'run_test3.sh' }
)
}

Solution

1. Step 1: Recall declarative pipeline structure for parallel stages

   Declarative pipelines require a stages block containing stage blocks; parallel tests go inside a single stage with a parallel block.

2. Step 2: Identify correct syntax for parallel tests with failure propagation

   stages { stage('Parallel Tests') { parallel { Test1: { steps { sh 'run_test1.sh' } } Test2: { steps { sh 'run_test2.sh' } } Test3: { steps { sh 'run_test3.sh' } } } } } correctly uses stages block with a stage named 'Parallel Tests' containing a parallel block where each parallel branch is a closure with steps and sh commands. This is the proper declarative syntax and ensures the build fails if any test fails.

3. Final Answer:

   stages { stage('Parallel Tests') { parallel { Test1: { steps { sh 'run_test1.sh' } } Test2: { steps { sh 'run_test2.sh' } } Test3: { steps { sh 'run_test3.sh' } } } } } -> Option A

4. Quick Check:

   Declarative parallel inside a stage within stages block with steps closures [OK]

Hint: Use parallel inside a stage within stages block with steps closures [OK]

Common Mistakes:

• Placing parallel outside stages block
• Using parallel without wrapping in a stage
• Mixing scripted and declarative syntax