Parallel stages execution in Jenkins - Commands & Configuration

Introduction

Sometimes you want to run multiple tasks at the same time to save time. Jenkins lets you run different parts of your build or test process in parallel. This helps finish work faster by using multiple workers at once.

When you want to run tests for different parts of your app at the same time to get results quicker

When you need to build multiple versions of your software for different platforms simultaneously

When you want to deploy to multiple environments at once without waiting for one to finish

When you want to run different code quality checks in parallel to speed up feedback

When you want to run multiple independent jobs that do not depend on each other

Config File - Jenkinsfile

Jenkinsfile

pipeline {
  agent any
  stages {
    stage('Parallel Example') {
      parallel {
        Build: {
          steps {
            echo 'Building the project'
            sh 'sleep 5'
          }
        },
        Test: {
          steps {
            echo 'Running tests'
            sh 'sleep 3'
          }
        },
        Lint: {
          steps {
            echo 'Checking code style'
            sh 'sleep 2'
          }
        }
      }
    }
  }
}

This Jenkinsfile defines a pipeline with one main stage called 'Parallel Example'. Inside it, three stages run at the same time: 'Build', 'Test', and 'Lint'. Each stage has simple commands that simulate work using sleep. This setup shows how Jenkins can run multiple tasks in parallel to save time.

Commands

This command updates the Jenkins job with the new Jenkinsfile that contains parallel stages. It tells Jenkins to use this pipeline definition.

Terminal

jenkins-jobs --conf jenkins.ini update Jenkinsfile

Expected Output

Job updated successfully

This command starts the Jenkins pipeline job named 'example-pipeline' and waits for it to finish. It runs the parallel stages defined in the Jenkinsfile.

Terminal

jenkins-cli build example-pipeline -s

Expected Output

[Pipeline] Start of Pipeline

[Pipeline] stage

[Pipeline] { (Parallel Example)

[Pipeline] parallel

[Pipeline] { (Build)

Building the project

[Pipeline] sh

[Pipeline] } // (Build)

[Pipeline] { (Test)

Running tests

[Pipeline] sh

[Pipeline] } // (Test)

[Pipeline] { (Lint)

Checking code style

[Pipeline] sh

[Pipeline] } // (Lint)

[Pipeline] } // parallel

[Pipeline] } // (Parallel Example)

[Pipeline] End of Pipeline

Finished: SUCCESS

→

-s - Waits for the job to finish and shows the console output

This command fetches the console output of the last run of the 'example-pipeline' job to verify the parallel stages ran correctly.

Terminal

jenkins-cli console example-pipeline

Expected Output

[Pipeline] Start of Pipeline

[Pipeline] stage

[Pipeline] { (Parallel Example)

[Pipeline] parallel

[Pipeline] { (Build)

Building the project

[Pipeline] sh

[Pipeline] } // (Build)

[Pipeline] { (Test)

Running tests

[Pipeline] sh

[Pipeline] } // (Test)

[Pipeline] { (Lint)

Checking code style

[Pipeline] sh

[Pipeline] } // (Lint)

[Pipeline] } // parallel

[Pipeline] } // (Parallel Example)

[Pipeline] End of Pipeline

Finished: SUCCESS

Key Concept

If you remember nothing else from this pattern, remember: Jenkins can run multiple stages at the same time using the 'parallel' block to save time and speed up your pipeline.

Common Mistakes

Placing 'parallel' outside of a 'stage' block

Jenkins requires 'parallel' to be inside a stage; otherwise, the pipeline syntax is invalid and will fail to run.

Always put the 'parallel' block inside a named 'stage' block in your Jenkinsfile.

Using the same stage name multiple times inside parallel

Stage names must be unique; duplicate names cause Jenkins to throw an error or overwrite stages.

Give each parallel stage a unique name to avoid conflicts.

Running long blocking commands without proper timeout

If a parallel stage hangs, it can block the entire pipeline and waste resources.

Use timeouts or proper error handling inside each parallel stage to avoid indefinite waits.

Summary

Define parallel stages inside a 'parallel' block within a Jenkins pipeline stage.

Use the Jenkins CLI to update the job and run the pipeline to see parallel execution.

Check the console output to verify that stages ran simultaneously and completed successfully.

Practice

1. What is the main purpose of using the parallel block in a Jenkins pipeline?

easy

A. To run multiple stages at the same time to save time

B. To run stages one after another in a sequence

C. To pause the pipeline until manual approval

D. To restart the pipeline from the beginning

Solution

1. Step 1: Understand the role of parallel block

   The parallel block allows multiple tasks or stages to run simultaneously in Jenkins.

2. Step 2: Compare with other options

   Options A, B, and C describe restart, sequential execution, and manual pause, which are unrelated to parallel execution.

3. Final Answer:

   To run multiple stages at the same time to save time -> Option A

4. Quick Check:

   Parallel stages = run together [OK]

Hint: Parallel means tasks run at the same time, not one by one [OK]

Common Mistakes:

• Thinking parallel means sequential execution
• Confusing parallel with manual approval
• Assuming parallel restarts the pipeline

2. Which of the following is the correct syntax to define parallel stages inside a Jenkins pipeline?

easy

A. stage('Build') { steps { echo 'Building' } parallel { stage('Test') { steps { echo 'Testing' } } } }

B. stage('Build') { parallel { stage('Test') { steps { echo 'Testing' } } } }

C. stage('Build') { steps { parallel { echo 'Building' } } }

D. parallel([ "Build": { steps { echo 'Building' } }, "Test": { steps { echo 'Testing' } } ])

Solution

1. Step 1: Recall correct parallel syntax

   The parallel block should use a map of branches, each containing steps.

2. Step 2: Analyze options

   parallel([ "Build": { steps { echo 'Building' } }, "Test": { steps { echo 'Testing' } } ]) correctly defines parallel branches. Options B, C, and D misuse the placement of parallel or stages.

3. Final Answer:

   parallel([ "Build": { steps { echo 'Building' } }, "Test": { steps { echo 'Testing' } } ]) -> Option D

4. Quick Check:

   parallel wraps stages directly [OK]

Hint: Parallel wraps multiple stages side by side, not inside one stage [OK]

Common Mistakes:

• Putting parallel inside steps block
• Nesting stages inside other stages incorrectly
• Using parallel with single stage only

3. Given this Jenkins pipeline snippet, what will be the output order of the echo statements?

pipeline {
  agent any
  stages {
    stage('Parallel Stage') {
      steps {
        parallel (
          "A": {
            echo 'Running A'
          },
          "B": {
            echo 'Running B'
          }
        )
      }
    }
  }
}

medium

A. Running A\nRunning B

B. Running B\nRunning A

C. Running A and Running B in any order or simultaneously

D. Syntax error due to nested stages inside parallel

Solution

1. Step 1: Understand parallel execution output

   Stages inside parallel run at the same time, so their output order is not guaranteed.

2. Step 2: Check syntax validity

   The syntax is valid for parallel branches inside a stage's steps block, so no syntax error occurs.

3. Final Answer:

   Running A and Running B in any order or simultaneously -> Option C

4. Quick Check:

   Parallel stages output order is unpredictable [OK]

Hint: Parallel means output order can vary, no fixed sequence [OK]

Common Mistakes:

• Assuming output order is always A then B
• Thinking nested stages cause syntax error here
• Expecting sequential output in parallel block

4. You wrote this Jenkins pipeline snippet but it fails with a syntax error:

pipeline {
  agent any
  stages {
    stage('Test') {
      steps {
        parallel {
          stage('Unit') {
            steps { echo 'Unit tests' }
          }
          stage('Integration') {
            steps { echo 'Integration tests' }
          }
        }
      }
    }
  }
}

What is the error and how to fix it?

medium

A. stage names inside parallel must be unique; rename stages

B. parallel block cannot contain nested stage blocks directly; use map syntax with branches

C. parallel block requires agent declaration inside each stage

D. steps block cannot contain echo commands

Solution

1. Step 1: Identify parallel block syntax rules

   The parallel directive expects a map of branch names to closures containing steps, not loose stage blocks inside {}.

2. Step 2: Fix the pipeline structure

   Change to parallel( "Unit": { echo 'Unit tests' }, "Integration": { echo 'Integration tests' } ) to fix the syntax error.

3. Final Answer:

   parallel block cannot contain nested stage blocks directly; use map syntax with branches -> Option B

4. Quick Check:

   parallel uses branch map syntax [OK]

Hint: parallel must be outside steps block, directly under stage [OK]

Common Mistakes:

• Placing parallel inside steps block
• Ignoring pipeline syntax hierarchy
• Assuming echo commands cause error

5. You want to run three tasks in parallel: 'Build', 'Test', and 'Deploy'. The 'Deploy' stage should only run if both 'Build' and 'Test' succeed. How do you structure the Jenkins pipeline to achieve this?

hard

A. Use parallel for 'Build' and 'Test' stages, then a separate 'Deploy' stage after parallel block

B. Put all three stages inside a single parallel block

C. Run 'Deploy' inside parallel block with 'Build' and 'Test' but add a condition to skip if others fail

D. Run 'Deploy' before 'Build' and 'Test' in parallel block

Solution

1. Step 1: Understand dependency requirements

   'Deploy' must wait for 'Build' and 'Test' to finish successfully, so it cannot run in parallel with them.

2. Step 2: Structure pipeline accordingly

   Run 'Build' and 'Test' inside a parallel block, then add a separate 'Deploy' stage after that block to run only if previous succeed.

3. Final Answer:

   Use parallel for 'Build' and 'Test' stages, then a separate 'Deploy' stage after parallel block -> Option A

4. Quick Check:

   Dependent stage runs after parallel block [OK]

Hint: Run dependent stages after parallel block, not inside it [OK]

Common Mistakes:

• Putting dependent stage inside parallel block
• Running deploy before build and test
• Trying to conditionally skip stages inside parallel