Multiple SCM repositories in Jenkins - Commands & Configuration

Introduction

Sometimes your project needs code from more than one place. Jenkins can get code from multiple source code repositories in one job. This helps when your app depends on different projects stored separately.

When your build needs code from two or more Git repositories at the same time.

When you want to combine a shared library repository with your main app repository in one Jenkins job.

When you have frontend and backend code in separate repositories but want to build them together.

When you want to pull configuration files from a different repository than your application code.

When you want to test changes that span multiple repositories in one pipeline.

Config File - Jenkinsfile

Jenkinsfile

pipeline {
  agent any
  stages {
    stage('Checkout Multiple Repos') {
      steps {
        script {
          // Checkout main app repository
          checkout([$class: 'GitSCM', branches: [[name: '*/main']],
            userRemoteConfigs: [[url: 'https://github.com/example-org/main-app.git']]])

          // Checkout shared library repository into 'libs' folder
          dir('libs') {
            checkout([$class: 'GitSCM', branches: [[name: '*/main']],
              userRemoteConfigs: [[url: 'https://github.com/example-org/shared-libs.git']]])
          }
        }
      }
    }
    stage('Build') {
      steps {
        echo 'Building the project using code from multiple repositories'
      }
    }
  }
}

This Jenkinsfile defines a pipeline with two stages. In the first stage, it checks out two Git repositories: the main app repository into the workspace root, and a shared library repository into a subfolder named 'libs'. This allows the build stage to use code from both places.

The checkout step uses the Git plugin to pull code. The dir('libs') block changes the folder before checking out the second repo.

Commands

This command updates or creates a Jenkins job named 'my-multi-scm-job' using a job definition file. It applies the configuration so Jenkins knows to run the pipeline with multiple SCM checkouts.

Terminal

jenkins-jobs --conf jenkins.ini update my-multi-scm-job.yaml

Expected Output

Job 'my-multi-scm-job' updated successfully.

This command triggers the Jenkins job named 'my-multi-scm-job' to start the pipeline. The '-s' flag waits for the build to finish and shows the console output.

Terminal

java -jar jenkins-cli.jar -s http://localhost:8080 build my-multi-scm-job -s

Expected Output

[Pipeline] Start of Pipeline

[Pipeline] stage

[Pipeline] { (Checkout Multiple Repos)

Cloning repository https://github.com/example-org/main-app.git

Cloning repository https://github.com/example-org/shared-libs.git

[Pipeline] dir

[Pipeline] checkout

[Pipeline] }

[Pipeline] stage

[Pipeline] { (Build)

Building the project using code from multiple repositories

[Pipeline] }

[Pipeline] End of Pipeline

Finished: SUCCESS

→

-s - Waits for the build to complete and shows the console output

This command fetches the console output of the last build of the job to verify the checkout and build steps ran correctly.

Terminal

java -jar jenkins-cli.jar -s http://localhost:8080 console my-multi-scm-job

Expected Output

[Pipeline] Start of Pipeline

[Pipeline] stage

[Pipeline] { (Checkout Multiple Repos)

Cloning repository https://github.com/example-org/main-app.git

Cloning repository https://github.com/example-org/shared-libs.git

[Pipeline] dir

[Pipeline] checkout

[Pipeline] }

[Pipeline] stage

[Pipeline] { (Build)

Building the project using code from multiple repositories

[Pipeline] }

[Pipeline] End of Pipeline

Finished: SUCCESS

Key Concept

If you remember nothing else from this pattern, remember: Jenkins can check out multiple source code repositories into different folders in one job using multiple checkout steps.

Common Mistakes

Trying to check out multiple repositories without specifying different folders for each.

This causes the second checkout to overwrite the first repository's files, leading to build failures.

Use the 'dir' step to change directories before each checkout so repositories do not overwrite each other.

Using multiple 'checkout' steps outside a 'script' block in declarative pipeline.

Declarative pipelines require multiple SCM checkouts inside a 'script' block to run Groovy code.

Wrap multiple checkout steps inside a 'script' block within the stage.

Summary

Use multiple 'checkout' steps inside a 'script' block to pull code from different repositories.

Use 'dir' to place each repository in its own folder to avoid overwriting files.

Trigger the Jenkins job and check console output to verify all repositories were checked out successfully.

Practice

1.

What is the main benefit of using multiple SCM repositories in a Jenkins job?

easy

A. It allows Jenkins to fetch code from several repositories in one build.

B. It speeds up the Jenkins server performance automatically.

C. It merges all repositories into a single repository permanently.

D. It restricts Jenkins to use only one branch per build.

Solution

1. Step 1: Understand Jenkins SCM usage

   Jenkins uses SCM to get source code for building projects.

2. Step 2: Identify multiple SCM benefit

   Multiple SCM lets Jenkins pull from many repositories in one job, useful for projects with dependencies.

3. Final Answer:

   It allows Jenkins to fetch code from several repositories in one build. -> Option A

4. Quick Check:

   Multiple SCM = fetch multiple repos [OK]

Hint: Multiple SCM means multiple code sources in one job [OK]

Common Mistakes:

• Thinking it merges repos permanently
• Assuming it improves server speed
• Believing it limits branches

2.

Which Jenkins pipeline syntax correctly defines two SCM repositories using checkout steps?

<pre>1) checkout([$class: 'GitSCM', userRemoteConfigs: [[url: 'https://repo1.git']]]) checkout([$class: 'GitSCM', userRemoteConfigs: [[url: 'https://repo2.git']]]) 2) checkout([$class: 'GitSCM', userRemoteConfigs: [[url: 'https://repo1.git', url: 'https://repo2.git']]]) 3) checkout([$class: 'GitSCM', url: 'https://repo1.git']) checkout([$class: 'GitSCM', url: 'https://repo2.git']) 4) checkout([$class: 'GitSCM', userRemoteConfigs: ['https://repo1.git', 'https://repo2.git']])

easy

A. Option 1

B. Option 2

C. Option 3

D. Option 4

Solution

1. Step 1: Check syntax for multiple SCM in Jenkins pipeline

   Each checkout step handles one SCM repo; multiple repos require multiple checkout calls.

2. Step 2: Analyze options

   Option 1 uses two separate checkout calls with correct userRemoteConfigs. Option 2 tries to combine repos in one config incorrectly. Option 3 uses wrong key url directly. Option 4 uses wrong list format.

3. Final Answer:

   Option 1 -> Option A

4. Quick Check:

   Separate checkout per repo = Option 1 [OK]

Hint: Use separate checkout steps for each repo [OK]

Common Mistakes:

• Combining multiple URLs in one userRemoteConfigs entry
• Using 'url' key instead of userRemoteConfigs
• Passing URLs as plain list without map

3.

Given this Jenkins pipeline snippet, what will be the result?

stage('Checkout') {
  steps {
    checkout([$class: 'GitSCM', userRemoteConfigs: [[url: 'https://repo1.git']]])
    checkout([$class: 'GitSCM', userRemoteConfigs: [[url: 'https://repo2.git']]])
  }
}

Choose the correct statement about the workspace after this stage runs.

medium

A. The workspace contains code from both repo1 and repo2 in separate folders.

B. The workspace contains only repo2 code, repo1 is overwritten.

C. The workspace contains merged files from repo1 and repo2 in the root folder.

D. The build will fail due to multiple checkout commands.

Solution

1. Step 1: Understand default checkout behavior

   By default, Jenkins checks out each repo into the same workspace folder, overwriting previous content.

2. Step 2: Analyze the effect of two checkout calls

   The second checkout replaces the first repo's files because both use the root workspace folder without specifying subdirectories.

3. Final Answer:

   The workspace contains only repo2 code, repo1 is overwritten. -> Option B

4. Quick Check:

   Multiple checkout without directories overwrites workspace [OK]

Hint: Without directories, last checkout overwrites workspace [OK]

Common Mistakes:

• Assuming repos merge automatically
• Thinking Jenkins creates subfolders by default
• Believing multiple checkouts cause build failure

4.

In a Jenkins job configured with multiple SCM repositories, the build fails with this error: hudson.plugins.git.GitException: Command "git clone" failed. What is the most likely cause?

medium

A. The workspace directory is not cleaned before checkout.

B. The Jenkins server has no internet connection.

C. The Jenkinsfile syntax is invalid.

D. The userRemoteConfigs list is empty or missing URLs.

Solution

1. Step 1: Understand the error context

   Git clone failure usually means the repository URL is invalid or missing.

2. Step 2: Check multiple SCM config

   If userRemoteConfigs is empty or missing URLs, Jenkins cannot clone repos, causing this error.

3. Final Answer:

   The userRemoteConfigs list is empty or missing URLs. -> Option D

4. Quick Check:

   Missing repo URLs cause git clone failure [OK]

Hint: Check repo URLs in userRemoteConfigs first [OK]

Common Mistakes:

• Blaming workspace cleanup for git clone errors
• Assuming Jenkinsfile syntax causes git clone failure
• Ignoring network issues as cause

5.

You want to configure a Jenkins pipeline to checkout two Git repositories into separate folders repo1 and repo2 inside the workspace. Which pipeline snippet correctly achieves this?

checkout([$class: 'GitSCM', userRemoteConfigs: [[url: 'https://repo1.git']], extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'repo1']]])
checkout([$class: 'GitSCM', userRemoteConfigs: [[url: 'https://repo2.git']], extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'repo2']]])

hard

A. This snippet merges both repos into the root workspace folder.

B. This snippet will fail because RelativeTargetDirectory is not supported.

C. This snippet correctly checks out both repos into separate folders.

D. This snippet only checks out the first repo; second is ignored.

Solution

1. Step 1: Understand RelativeTargetDirectory extension

   This extension tells Jenkins to checkout the repo into a specified subfolder inside the workspace.

2. Step 2: Analyze the snippet

   Each checkout uses RelativeTargetDirectory with different folder names, so repos go into separate folders without overwriting.

3. Final Answer:

   This snippet correctly checks out both repos into separate folders. -> Option C

4. Quick Check:

   RelativeTargetDirectory separates repos in workspace [OK]

Hint: Use RelativeTargetDirectory to separate repos in workspace [OK]

Common Mistakes:

• Not using RelativeTargetDirectory causes overwriting
• Assuming Jenkins merges repos automatically
• Believing second checkout is ignored