Kubernetes agents in Jenkins - Deep Dive

Overview - Kubernetes agents

What is it?

Kubernetes agents are worker nodes or pods that run tasks for Jenkins pipelines inside a Kubernetes cluster. They allow Jenkins to dynamically create and manage build environments on demand. This means Jenkins can run jobs in isolated containers that match the needs of each task.

Why it matters

Without Kubernetes agents, Jenkins would need fixed, pre-configured machines to run jobs, which wastes resources and limits flexibility. Kubernetes agents let Jenkins scale up and down automatically, saving costs and speeding up builds. This dynamic approach fits modern cloud-native workflows and helps teams deliver software faster.

Where it fits

Before learning Kubernetes agents, you should understand basic Jenkins concepts like pipelines and agents, and have a grasp of Kubernetes fundamentals like pods and clusters. After this, you can explore advanced Jenkins-Kubernetes integrations, custom pod templates, and scaling strategies.

Mental Model

Core Idea

Kubernetes agents are temporary, on-demand containers that Jenkins creates inside a Kubernetes cluster to run build tasks efficiently and isolated.

Think of it like...

Imagine a busy kitchen where chefs (Jenkins) call for special cooking stations (Kubernetes agents) only when needed. Each station is set up with the exact tools and ingredients for a recipe, then cleaned up after cooking, freeing space for the next order.

Jenkins Master
  │
  ├─ Requests build
  │
  ▼
Kubernetes Cluster
  ├─ Pod 1 (Agent for Job A)
  ├─ Pod 2 (Agent for Job B)
  └─ Pod N (Agent for Job N)

Each pod runs a container with the build environment
and disappears after the job finishes.

Build-Up - 7 Steps

1

Foundation

What is a Jenkins Agent?

Concept: Introduce the basic idea of Jenkins agents as workers that run jobs.

Jenkins uses agents to run parts of your build or deployment process. These agents can be physical machines, virtual machines, or containers. They take instructions from the Jenkins master and execute tasks like compiling code or running tests.

Result

You understand that Jenkins agents are the workers that do the actual job work, separate from the main Jenkins controller.

Knowing that Jenkins separates control (master) from work (agents) helps you see why agents are essential for scaling and isolating builds.

2

Foundation

Basics of Kubernetes Pods

Concept: Explain what a Kubernetes pod is and how it runs containers.

A pod is the smallest unit in Kubernetes that can run one or more containers. Pods share storage and network, and they run on nodes inside the cluster. Pods can be created and destroyed dynamically based on demand.

Result

You grasp that pods are temporary containers running inside Kubernetes, ready to do work.

Understanding pods as temporary, isolated environments is key to seeing how Jenkins agents can be pods.

3

Intermediate

Jenkins Kubernetes Plugin Overview

Concept: Introduce the Jenkins Kubernetes plugin that connects Jenkins with Kubernetes to create agents.

The Jenkins Kubernetes plugin lets Jenkins talk to a Kubernetes cluster. When a job needs to run, Jenkins asks Kubernetes to create a pod with a container that has the right tools. After the job finishes, the pod is deleted automatically.

Result

You see how Jenkins can dynamically create build environments inside Kubernetes without manual setup.

Knowing this plugin automates agent creation helps you appreciate the power of dynamic scaling and environment consistency.

4

Intermediate

Pod Templates and Container Specs

🤔Before reading on: do you think a pod template defines the entire pod or just the container image? Commit to your answer.

Concept: Explain how pod templates define the shape and tools of the agent pods Jenkins creates.

Pod templates describe what containers run inside the pod, what images they use, resource limits, environment variables, and volumes. Jenkins uses these templates to create pods that fit the job's needs exactly.

Result

You understand how to customize the build environment by changing pod templates.

Knowing pod templates control the agent's environment lets you tailor builds precisely and avoid conflicts.

5

Intermediate

Agent Lifecycle in Kubernetes

🤔Before reading on: do you think Kubernetes agents stay alive after the job completes? Commit to your answer.

Concept: Describe how Jenkins agents are created, used, and destroyed in Kubernetes.

When Jenkins starts a job, it requests Kubernetes to create a pod from the template. The pod runs the job, then terminates and is deleted. This keeps the cluster clean and resources free.

Result

You see that agents are temporary and only exist while needed.

Understanding the ephemeral nature of agents prevents resource leaks and helps design efficient pipelines.

6

Advanced

Scaling and Resource Management

🤔Before reading on: do you think Kubernetes agents can share resources or must each have dedicated CPU and memory? Commit to your answer.

Concept: Explain how Kubernetes manages resources for agents and how Jenkins can scale builds.

Kubernetes schedules pods based on resource requests and limits. Jenkins can run many agents in parallel, and Kubernetes ensures they don't exceed node capacity. You can set resource limits in pod templates to avoid overloading nodes.

Result

You understand how to control build concurrency and resource use in a cluster.

Knowing resource management helps prevent slow builds and cluster crashes under heavy load.

7

Expert

Security and Network Isolation

🤔Before reading on: do you think all Jenkins agents share the same network identity inside Kubernetes? Commit to your answer.

Concept: Discuss advanced security practices for Kubernetes agents, including network policies and credentials.

Each agent pod runs isolated with its own network identity. You can apply Kubernetes network policies to restrict communication. Secrets and credentials are injected securely into pods. This isolation protects sensitive data and limits attack surfaces.

Result

You see how to secure Jenkins agents in multi-tenant or sensitive environments.

Understanding security isolation prevents accidental data leaks and strengthens your CI/CD pipeline defenses.

Under the Hood

When Jenkins triggers a build, the Kubernetes plugin sends a request to the Kubernetes API server to create a pod based on a pod template. Kubernetes schedules the pod on a node, pulls the container image, and starts the container. The container runs the Jenkins agent process, which connects back to the Jenkins master. After the job finishes, the pod is terminated and deleted by Kubernetes, freeing resources.

Why designed this way?

This design leverages Kubernetes' native ability to manage containers and resources dynamically. It avoids the need for permanent build machines, reduces idle resource waste, and ensures build environments are consistent and isolated. Alternatives like static agents or VM-based agents are less flexible and more costly.

Jenkins Master
  │
  ├─ Kubernetes Plugin
  │    │
  │    └─ API Request to Kubernetes
  │          │
  │          ▼
  │      Kubernetes API Server
  │          │
  │          ▼
  │      Scheduler assigns Pod to Node
  │          │
  │          ▼
  │      Node runs Pod with Jenkins Agent Container
  │          │
  │          ▼
  └─ Agent connects back to Jenkins Master

After job:
  Pod terminated and deleted by Kubernetes

Myth Busters - 4 Common Misconceptions

Quick: Do Kubernetes agents persist after a Jenkins job finishes? Commit to yes or no.

Common Belief: Kubernetes agents stay running after the job to speed up future builds.

Reality: Kubernetes agents are ephemeral and deleted after the job completes to save resources.

Why it matters: Assuming agents persist can lead to resource exhaustion and unexpected build failures.

Quick: Do all Jenkins agents share the same container image in Kubernetes? Commit to yes or no.

Common Belief: All Kubernetes agents use the same container image regardless of job requirements.

Reality: Pod templates allow different container images per job, enabling customized environments.

Why it matters: Believing all agents are identical limits flexibility and can cause build errors due to missing tools.

Quick: Can Jenkins master run build jobs directly without agents? Commit to yes or no.

Common Belief: Jenkins master runs all build jobs itself without needing agents.

Reality: Jenkins master delegates build jobs to agents to keep control and workload separate.

Why it matters: Running builds on the master risks stability and security of the Jenkins server.

Quick: Are Kubernetes agents automatically secure without extra configuration? Commit to yes or no.

Common Belief: Kubernetes agents are secure by default and need no additional network or credential setup.

Reality: Security requires explicit network policies and secret management to protect agents and data.

Why it matters: Ignoring security can expose sensitive credentials and allow unauthorized access.

Expert Zone

1

Pod templates can include multiple containers, such as sidecars for logging or monitoring, which many users overlook.

2

Resource requests and limits in pod templates affect Kubernetes scheduling and build performance in subtle ways.

3

Agent pods can be configured with node selectors or tolerations to run on specific nodes, enabling specialized hardware use.

When NOT to use

Kubernetes agents are not ideal if your Jenkins environment is very small or runs on-premises without Kubernetes. In such cases, static agents or VM-based agents may be simpler. Also, if builds require very long-lived environments, ephemeral pods might not fit well.

Production Patterns

In production, teams use pod templates stored as code for version control, combine Kubernetes agents with persistent storage for caching, and integrate network policies for multi-tenant security. They also monitor agent pod health and scale Kubernetes clusters automatically based on build queue length.

Connections

Serverless Computing

Both use ephemeral, on-demand compute resources to run tasks without managing servers.

Understanding Kubernetes agents as serverless workers helps grasp how modern cloud platforms optimize resource use and cost.

Factory Assembly Line

Kubernetes agents act like flexible workstations that appear only when a product needs a specific step done.

Seeing agents as temporary stations clarifies how dynamic resource allocation improves efficiency.

Operating System Process Scheduling

Kubernetes schedules pods like an OS schedules processes, balancing load and resources.

Knowing this analogy helps understand how Kubernetes manages agent pods fairly and efficiently.

Common Pitfalls

#1 Forgetting to configure pod templates correctly causes build failures.

Wrong approach:

podTemplate {
containers {
containerTemplate(name: 'jnlp', image: 'wrong-image')
}
}

Correct approach:

podTemplate {
containers {
containerTemplate(name: 'jnlp', image: 'jenkins/inbound-agent:latest')
}
}

Root cause:

Misunderstanding that the agent container image must match Jenkins requirements.

#2 Not setting resource limits leads to cluster overload.

Wrong approach:

podTemplate {
containers {
containerTemplate(name: 'jnlp', image: 'jenkins/inbound-agent:latest')
}
}

Correct approach:

podTemplate {
containers {
containerTemplate(name: 'jnlp', image: 'jenkins/inbound-agent:latest', resourceRequestCpu: '500m', resourceRequestMemory: '1Gi')
}
}

Root cause:

Ignoring Kubernetes resource management concepts.

#3 Assuming agents run on the Jenkins master node causes confusion.

Wrong approach:

Running builds without specifying Kubernetes agents, expecting them on master.

Correct approach:

Configure Jenkins pipeline with 'agent { kubernetes { ... } }' to run builds in Kubernetes pods.

Root cause:

Not understanding Jenkins master-agent separation and Kubernetes integration.

Key Takeaways

Kubernetes agents let Jenkins run build jobs in temporary, isolated containers inside a Kubernetes cluster.

Pod templates define the exact environment and resources each agent pod uses, enabling flexible and consistent builds.

Agents are created on demand and deleted after use, which saves resources and keeps the cluster clean.

Proper resource and security configurations are essential to avoid build failures and protect sensitive data.

Understanding Kubernetes agents bridges Jenkins automation with cloud-native infrastructure for scalable CI/CD.

Practice

1. What is the main purpose of using Kubernetes agents in Jenkins?

easy

A. To run Jenkins jobs inside isolated pods on a Kubernetes cluster

B. To store Jenkins job logs permanently on the master node

C. To replace Jenkins pipelines with Kubernetes scripts

D. To run Jenkins jobs only on physical machines

Solution

1. Step 1: Understand Kubernetes agents role

   Kubernetes agents run Jenkins jobs inside pods, providing isolation and scalability.

2. Step 2: Compare options

   Only To run Jenkins jobs inside isolated pods on a Kubernetes cluster correctly describes running jobs inside pods on Kubernetes clusters.

3. Final Answer:

   To run Jenkins jobs inside isolated pods on a Kubernetes cluster -> Option A

4. Quick Check:

   Kubernetes agents = isolated pods for Jenkins jobs [OK]

Hint: Kubernetes agents run jobs inside pods, not on master node [OK]

Common Mistakes:

• Thinking agents run jobs on master node
• Confusing Kubernetes agents with storage solutions
• Assuming Kubernetes replaces Jenkins pipelines

2. Which Jenkins pipeline syntax correctly defines a Kubernetes agent using podTemplate?

easy

A. containerTemplate(name: 'jnlp', image: 'jenkins/inbound-agent') { podTemplate { node { /* steps */ } } }

B. node { podTemplate(containers: [containerTemplate(name: 'jnlp', image: 'jenkins/inbound-agent')]) { /* steps */ } }

C. podTemplate(containers: [containerTemplate(name: 'jnlp', image: 'jenkins/inbound-agent')]) { node { /* steps */ } }

D. podTemplate { node(containers: ['jnlp']) { /* steps */ } }

Solution

1. Step 1: Identify correct podTemplate usage

   The podTemplate block wraps the node block and defines containers properly.

2. Step 2: Validate syntax correctness

   podTemplate(containers: [containerTemplate(name: 'jnlp', image: 'jenkins/inbound-agent')]) { node { /* steps */ } } correctly places podTemplate outside node and defines containerTemplate properly.

3. Final Answer:

   podTemplate(containers: [containerTemplate(name: 'jnlp', image: 'jenkins/inbound-agent')]) { node { /* steps */ } } -> Option C

4. Quick Check:

   podTemplate wraps node with containerTemplate [OK]

Hint: podTemplate wraps node, containerTemplate inside podTemplate [OK]

Common Mistakes:

• Placing podTemplate inside node block
• Using containerTemplate outside podTemplate
• Incorrect nesting of blocks

3. Given this Jenkins pipeline snippet, what will be the output if the pod agent is correctly configured?

podTemplate(containers: [containerTemplate(name: 'jnlp', image: 'jenkins/inbound-agent')]) {
  node {
    stage('Test') {
      echo 'Running inside Kubernetes agent'
    }
  }
}

medium

A. The pipeline will fail because podTemplate is missing a label

B. The message 'Running inside Kubernetes agent' will be printed in the Jenkins console log

C. Nothing will print because echo is not supported inside podTemplate

D. The pipeline will run on the Jenkins master node ignoring podTemplate

Solution

1. Step 1: Understand podTemplate and node behavior

   The podTemplate defines the Kubernetes pod agent, and node runs the steps inside it.

2. Step 2: Analyze echo command inside stage

   The echo command prints the message to the Jenkins console log during the 'Test' stage.

3. Final Answer:

   The message 'Running inside Kubernetes agent' will be printed in the Jenkins console log -> Option B

4. Quick Check:

   echo inside podTemplate prints message [OK]

Hint: echo prints messages in Jenkins console inside podTemplate [OK]

Common Mistakes:

• Assuming echo won't work inside podTemplate
• Thinking podTemplate requires label always
• Confusing podTemplate with running on master node

4. You wrote this Jenkins pipeline but it fails with "No nodes available" error:

podTemplate(containers: [containerTemplate(name: 'jnlp', image: 'jenkins/inbound-agent')]) {
  stage('Build') {
    node {
      echo 'Building...'
    }
  }
}

What is the likely cause?

medium

A. The containerTemplate name 'jnlp' is invalid

B. The stage block is outside the node block

C. The node block is outside the podTemplate block

D. The node block is missing a label matching the podTemplate

Solution

1. Step 1: Check node block label requirement

   When using podTemplate, the node block must specify the label matching the podTemplate to allocate the pod.

2. Step 2: Identify missing label causes no node available

   Without label, Jenkins cannot find the Kubernetes agent pod, causing the error.

3. Final Answer:

   The node block is missing a label matching the podTemplate -> Option D

4. Quick Check:

   node label missing = no Kubernetes pod allocated [OK]

Hint: Always specify node label matching podTemplate to avoid no nodes error [OK]

Common Mistakes:

• Not specifying node label
• Misplacing node and stage blocks
• Assuming containerTemplate name causes error

5. You want to run a Jenkins job on a Kubernetes agent with two containers: one for building with Maven and one for running tests with Node.js. Which pipeline snippet correctly defines this setup?

hard

A. podTemplate(containers: [containerTemplate(name: 'maven', image: 'maven:3.8.1'), containerTemplate(name: 'nodejs', image: 'node:16')]) {
node {
stage('Build') {
container('maven') {
sh 'mvn clean install'
}
}
stage('Test') {
container('nodejs') {
sh 'npm test'
}
}
}
}

B. podTemplate(containers: [containerTemplate(name: 'maven', image: 'maven:3.8.1')]) {
node {
stage('Build') {
sh 'mvn clean install'
}
stage('Test') {
container('nodejs') {
sh 'npm test'
}
}
}
}

C. podTemplate(containers: [containerTemplate(name: 'nodejs', image: 'node:16')]) {
node {
stage('Build') {
container('maven') {
sh 'mvn clean install'
}
}
stage('Test') {
sh 'npm test'
}
}
}

D. podTemplate(containers: [containerTemplate(name: 'maven', image: 'maven:3.8.1'), containerTemplate(name: 'nodejs', image: 'node:16')]) {
node {
stage('Build') {
sh 'mvn clean install'
}
stage('Test') {
sh 'npm test'
}
}
}

Solution

1. Step 1: Define multiple containers in podTemplate

   podTemplate(containers: [containerTemplate(name: 'maven', image: 'maven:3.8.1'), containerTemplate(name: 'nodejs', image: 'node:16')]) { node { stage('Build') { container('maven') { sh 'mvn clean install' } } stage('Test') { container('nodejs') { sh 'npm test' } } } } correctly defines two containers: 'maven' and 'nodejs' with proper images.

2. Step 2: Use container block to run commands in specific containers

   podTemplate(containers: [containerTemplate(name: 'maven', image: 'maven:3.8.1'), containerTemplate(name: 'nodejs', image: 'node:16')]) { node { stage('Build') { container('maven') { sh 'mvn clean install' } } stage('Test') { container('nodejs') { sh 'npm test' } } } } uses container('name') to run build and test commands inside respective containers.

3. Final Answer:

   Option A correctly defines two containers and uses container blocks for commands -> Option A

4. Quick Check:

   Multiple containers + container blocks = podTemplate(containers: [containerTemplate(name: 'maven', image: 'maven:3.8.1'), containerTemplate(name: 'nodejs', image: 'node:16')]) { node { stage('Build') { container('maven') { sh 'mvn clean install' } } stage('Test') { container('nodejs') { sh 'npm test' } } } } [OK]

Hint: Use container('name') block to run commands in specific pod containers [OK]

Common Mistakes:

• Not defining all containers in podTemplate
• Running commands outside container blocks
• Mixing container names or missing container blocks