Cloud agent provisioning (EC2, Azure) in Jenkins - Commands & Configuration

Introduction

When you want Jenkins to run jobs on cloud servers, you need to create agents in the cloud automatically. This process is called cloud agent provisioning. It saves time by creating and removing servers only when needed.

When you want Jenkins to run builds on Amazon EC2 instances without manual setup.

When you need Jenkins agents to scale up and down automatically in Azure based on workload.

When you want to avoid keeping permanent build servers and save costs by using cloud resources on demand.

When you want to run different build environments for different projects using cloud agents.

When you want Jenkins to manage cloud servers lifecycle for running jobs.

Config File - Jenkinsfile

Jenkinsfile

pipeline {
  agent {
    label 'ec2-agent'
  }
  stages {
    stage('Build') {
      steps {
        echo 'Running build on EC2 agent'
      }
    }
  }
}

This Jenkinsfile defines a pipeline that runs on a cloud agent labeled 'ec2-agent' from the AWS EC2 cloud configuration. The agent block tells Jenkins to use an agent with the label 'ec2-agent', which should be configured to provision an EC2 instance automatically. The stages section runs the build steps on that cloud agent.

Commands

This command installs the Amazon EC2 plugin in Jenkins, which allows Jenkins to create and manage EC2 agents automatically.

Terminal

jenkins-cli install-plugin amazon-ec2

Expected Output

Installing plugin: amazon-ec2

Success

Restarts Jenkins safely to apply the new plugin without interrupting running jobs.

Terminal

jenkins-cli safe-restart

Expected Output

Jenkins is restarting...

Adds AWS credentials to Jenkins from an XML file so Jenkins can authenticate to AWS and provision EC2 instances.

Terminal

jenkins-cli create-credentials-by-xml system::system::jenkins _ < aws-credentials.xml

Expected Output

Credentials added successfully

Runs a Groovy script to configure the EC2 cloud in Jenkins with details like region, AMI, instance type, and credentials.

Terminal

jenkins-cli groovy = < configure-ec2-cloud.groovy

Expected Output

EC2 cloud configured successfully

Starts a Jenkins pipeline job that uses the EC2 cloud agent to run the build on a provisioned EC2 instance.

Terminal

jenkins-cli build example-pipeline

Expected Output

[Pipeline] Start of Pipeline

[Pipeline] node

Running build on EC2 agent

[Pipeline] End of Pipeline

Finished: SUCCESS

Key Concept

If you remember nothing else from this pattern, remember: Jenkins can create and destroy cloud servers automatically to run jobs, saving time and resources.

Common Mistakes

Not adding AWS credentials to Jenkins before configuring EC2 cloud.

Jenkins cannot authenticate to AWS and will fail to create EC2 agents.

Add valid AWS credentials in Jenkins credentials store before configuring the EC2 cloud.

Using incorrect AMI ID or instance type in EC2 cloud configuration.

Jenkins will fail to launch EC2 instances or launch wrong environments.

Use valid AMI IDs and instance types that exist in your AWS region.

Not restarting Jenkins after installing the EC2 plugin.

The plugin will not be active and Jenkins cannot provision EC2 agents.

Always restart Jenkins safely after installing new plugins.

Summary

Install the Amazon EC2 plugin in Jenkins to enable cloud agent provisioning.

Add AWS credentials to Jenkins so it can authenticate to AWS.

Configure the EC2 cloud in Jenkins with region, AMI, and instance details.

Use a Jenkins pipeline that specifies the cloud agent label to run jobs on EC2 instances.

Jenkins will create and remove EC2 agents automatically to run your builds.