Cloud agent provisioning (EC2, Azure) in Jenkins - Mini Project: Build & Apply

Cloud Agent Provisioning with Jenkins on EC2 and Azure

📖 Scenario: You are setting up Jenkins to automatically provision cloud agents on AWS EC2 and Microsoft Azure. This helps Jenkins run jobs on fresh cloud machines that start when needed and stop when done, saving resources and cost.We will create a simple Jenkins pipeline script that defines the cloud agents and provisions them on demand.

🎯 Goal: Build a Jenkins pipeline script that defines cloud agents for EC2 and Azure, configures their settings, and runs a simple job on each agent.This teaches how to automate cloud agent provisioning using Jenkins declarative pipeline syntax.

📋 What You'll Learn

Create a Jenkins pipeline script with cloud agent definitions

Add configuration variables for EC2 and Azure agents

Use the agent directive to specify cloud agents

Run simple shell commands on each cloud agent

Print the output of the commands to verify provisioning

💡 Why This Matters

🌍 Real World

Automating cloud agent provisioning helps teams run CI/CD jobs on fresh cloud machines, improving scalability and cost efficiency.

💼 Career

DevOps engineers often configure Jenkins to provision and manage cloud agents on AWS and Azure for automated build and deployment pipelines.

1

Define EC2 and Azure cloud agents

Create a Jenkins pipeline script with a pipeline block. Inside agent, define two cloud agents: ec2-agent and azure-agent. Use label to name them exactly as 'ec2-agent' and 'azure-agent'.

Jenkins

pipeline {
    agent none
    stages {
        // Define cloud agents here
    }
}
# Your code here

Hint

Use agent none at pipeline level, then define agents inside parallel stages with agent { label '...' }.

2

Add configuration variables for cloud agents

Inside the pipeline block but before stages, add an environment block. Define two variables: EC2_INSTANCE_TYPE set to 't2.micro' and AZURE_VM_SIZE set to 'Standard_B1s'.

Jenkins

pipeline {
    agent none
    environment {
        // Define EC2_INSTANCE_TYPE and AZURE_VM_SIZE here
    }
    stages {
        stage('Setup Agents') {
            parallel {
                stage('EC2 Agent') {
                    agent { label 'ec2-agent' }
                    steps {
                        echo 'EC2 agent ready'
                    }
                }
                stage('Azure Agent') {
                    agent { label 'azure-agent' }
                    steps {
                        echo 'Azure agent ready'
                    }
                }
            }
        }
    }
}
# Your code here

Hint

Use the environment block to set variables inside the pipeline block.

3

Run shell commands on each cloud agent

Inside the steps of EC2 Agent stage, add a shell command to print echo "Running on EC2 instance type: $EC2_INSTANCE_TYPE". Inside the steps of Azure Agent stage, add a shell command to print echo "Running on Azure VM size: $AZURE_VM_SIZE".

Jenkins

pipeline {
    agent none
    environment {
        EC2_INSTANCE_TYPE = 't2.micro'
        AZURE_VM_SIZE = 'Standard_B1s'
    }
    stages {
        stage('Setup Agents') {
            parallel {
                stage('EC2 Agent') {
                    agent { label 'ec2-agent' }
                    steps {
                        // Add shell command to print EC2 instance type
                    }
                }
                stage('Azure Agent') {
                    agent { label 'azure-agent' }
                    steps {
                        // Add shell command to print Azure VM size
                    }
                }
            }
        }
    }
}
# Your code here

Hint

Use sh 'echo "text"' inside steps to run shell commands on agents.

4

Print the output to verify provisioning

Add a final stage called Verify Output that runs on agent any. Inside steps, add a shell command to print echo "Cloud agents provisioning test complete". This confirms the pipeline ran successfully.

Jenkins

pipeline {
    agent none
    environment {
        EC2_INSTANCE_TYPE = 't2.micro'
        AZURE_VM_SIZE = 'Standard_B1s'
    }
    stages {
        stage('Setup Agents') {
            parallel {
                stage('EC2 Agent') {
                    agent { label 'ec2-agent' }
                    steps {
                        sh 'echo "Running on EC2 instance type: $EC2_INSTANCE_TYPE"'
                    }
                }
                stage('Azure Agent') {
                    agent { label 'azure-agent' }
                    steps {
                        sh 'echo "Running on Azure VM size: $AZURE_VM_SIZE"'
                    }
                }
            }
        }
        // Add Verify Output stage here
    }
}
# Your code here

Hint

Add a new stage with agent any and a shell command to print the confirmation message.

Practice

1. What is the main purpose of cloud agent provisioning in Jenkins?

easy

A. To dynamically create and use cloud machines as Jenkins agents

B. To manually configure Jenkins agents on local machines

C. To backup Jenkins configuration files to the cloud

D. To monitor Jenkins job execution times

Solution

1. Step 1: Understand Jenkins agent provisioning

   Jenkins can create agents dynamically in the cloud to run jobs without manual setup.

2. Step 2: Identify the purpose of cloud provisioning

   Cloud agent provisioning automates creating cloud machines (like EC2 or Azure VMs) as Jenkins agents.

3. Final Answer:

   To dynamically create and use cloud machines as Jenkins agents -> Option A

4. Quick Check:

   Cloud agent provisioning = dynamic cloud agents [OK]

Hint: Cloud provisioning means automatic cloud agent creation [OK]

Common Mistakes:

• Confusing provisioning with manual setup
• Thinking it is for backup or monitoring
• Assuming it only works on local machines

2. Which of the following is the correct way to specify an EC2 AMI ID in Jenkins cloud agent configuration?

easy

A. ami-0abcdef1234567890

B. ec2-ami-123456

C. AMI:1234567890

D. image-ec2-abcdef

Solution

1. Step 1: Recognize EC2 AMI ID format

   EC2 AMI IDs start with 'ami-' followed by alphanumeric characters, e.g., 'ami-0abcdef1234567890'.

2. Step 2: Match the correct format

   ami-0abcdef1234567890 matches the standard EC2 AMI ID format exactly.

3. Final Answer:

   ami-0abcdef1234567890 -> Option A

4. Quick Check:

   EC2 AMI ID starts with 'ami-' [OK]

Hint: EC2 AMI IDs always start with 'ami-' [OK]

Common Mistakes:

• Using incorrect prefixes like 'ec2-ami-'
• Confusing AMI ID with image names
• Adding extra labels or colons

3. Given this Jenkins EC2 cloud agent snippet:

instanceCapStr: '3',
idleTerminationMinutes: 10,
ami: 'ami-0abc123def456',
remoteFS: '/home/jenkins'

What happens when 4 jobs are triggered simultaneously?

medium

A. Jenkins throws an error due to instance cap mismatch

B. 4 agents will be created ignoring the instance cap

C. All 4 jobs run on a single agent

D. Only 3 agents will be created; the 4th job waits in queue

Solution

1. Step 1: Understand instanceCapStr meaning

   The 'instanceCapStr' limits the max number of EC2 agents Jenkins can create to 3.

2. Step 2: Analyze job triggering behavior

   When 4 jobs start, Jenkins creates only 3 agents due to the cap; the 4th job waits in the queue.

3. Final Answer:

   Only 3 agents will be created; the 4th job waits in queue -> Option D

4. Quick Check:

   Instance cap limits agents = jobs queue if exceeded [OK]

Hint: Instance cap limits agents; extra jobs queue [OK]

Common Mistakes:

• Assuming Jenkins ignores instance cap
• Thinking all jobs run on one agent
• Expecting an error when cap exceeded

4. You configured an Azure cloud agent in Jenkins but agents never start. Which is the most likely cause?

medium

A. Instance cap set to zero

B. Jenkins master is offline

C. Incorrect Azure VM image name or ID

D. Idle timeout set too high

Solution

1. Step 1: Check Azure VM image configuration

   If the VM image name or ID is wrong, Jenkins cannot provision agents on Azure.

2. Step 2: Evaluate other options

   Jenkins master offline stops all jobs, but question focuses on agent provisioning. Instance cap zero means no agents, but usually agents would not start at all. Idle timeout affects termination, not start.

3. Final Answer:

   Incorrect Azure VM image name or ID -> Option C

4. Quick Check:

   Wrong VM image blocks agent start [OK]

Hint: Check VM image name first if agents don't start [OK]

Common Mistakes:

• Ignoring VM image correctness
• Blaming idle timeout for start issues
• Assuming Jenkins master offline without checking

5. You want to optimize Jenkins cloud agents on EC2 to reduce cost. Which combination of settings is best?

hard

A. Set instance cap high and idle termination to 60 minutes

B. Set instance cap low and idle termination to 5 minutes

C. Disable instance cap and idle termination

D. Set instance cap low and disable idle termination

Solution

1. Step 1: Understand instance cap effect

   A low instance cap limits max running agents, controlling cloud costs.

2. Step 2: Understand idle termination effect

   Setting idle termination to 5 minutes stops unused agents quickly, saving cost.

3. Step 3: Evaluate other options

   High cap or disabling termination increases cost. Disabling termination wastes resources.

4. Final Answer:

   Set instance cap low and idle termination to 5 minutes -> Option B

5. Quick Check:

   Low cap + quick termination = cost saving [OK]

Hint: Low cap + short idle timeout saves cloud cost [OK]

Common Mistakes:

• Setting high caps wastes money
• Disabling idle termination keeps idle agents running
• Ignoring cost impact of settings