Artifact retention policies in Jenkins - Deep Dive

Overview - Artifact retention policies

What is it?

Artifact retention policies are rules that decide how long build outputs, called artifacts, are kept in Jenkins. Artifacts are files like compiled programs or reports created during a build. These policies help manage storage by automatically deleting old or unnecessary artifacts. This keeps the Jenkins environment clean and efficient.

Why it matters

Without artifact retention policies, Jenkins would store every artifact forever, quickly using up disk space and slowing down the system. This can cause failures or delays in builds and increase maintenance work. Proper retention policies save storage, reduce costs, and keep the build system reliable and fast.

Where it fits

Before learning artifact retention policies, you should understand Jenkins basics like jobs, builds, and artifacts. After this, you can explore advanced Jenkins maintenance, storage optimization, and pipeline automation techniques.

Mental Model

Core Idea

Artifact retention policies are automated rules that decide when to keep or delete build files to save space and keep Jenkins healthy.

Think of it like...

It's like cleaning out your fridge regularly: you keep fresh food you need and throw away old leftovers to make room for new groceries.

┌───────────────────────────────┐
│         Jenkins Build          │
│  ┌───────────────┐            │
│  │  Artifacts    │            │
│  └──────┬────────┘            │
│         │                    │
│  ┌──────▼────────┐           │
│  │Retention Rules│           │
│  └──────┬────────┘           │
│         │                    │
│  ┌──────▼────────┐           │
│  │ Keep or Delete│           │
│  └───────────────┘           │
└───────────────────────────────┘

Build-Up - 7 Steps

1

Foundation

What are Jenkins artifacts

Concept: Introduce what artifacts are in Jenkins and why they matter.

In Jenkins, an artifact is any file or set of files produced by a build job. For example, a compiled program, a test report, or a zipped package. Artifacts are saved so you can use them later, like deploying or sharing results.

Result

You understand that artifacts are build outputs stored by Jenkins for future use.

Knowing what artifacts are helps you see why managing their storage is important.

2

Foundation

Why artifact retention is needed

Concept: Explain the problem of unlimited artifact storage.

Every build can create artifacts, and if Jenkins keeps all of them forever, disk space fills up fast. This slows down Jenkins and can cause failures. Retention policies prevent this by deleting old or unneeded artifacts automatically.

Result

You see the need for rules to control how long artifacts stay in Jenkins.

Understanding the storage problem motivates the use of retention policies.

3

Intermediate

Configuring retention policies in Jenkins

🤔Before reading on: do you think retention policies are set globally or per job? Commit to your answer.

Concept: Learn how to set artifact retention rules in Jenkins job configurations.

In Jenkins, you configure retention policies inside each job's settings under 'Discard old builds'. You can specify how many builds or days to keep. For example, keep artifacts for the last 10 builds or 30 days. Jenkins then deletes artifacts from older builds automatically.

Result

You can set rules to keep or delete artifacts based on build count or age.

Knowing that retention is per job lets you customize policies for different projects.

4

Intermediate

Retention policy options and criteria

🤔Before reading on: do you think Jenkins can delete artifacts but keep build records? Commit to your answer.

Concept: Explore different criteria Jenkins uses to decide artifact deletion.

Jenkins allows you to discard builds entirely or just their artifacts. You can keep build records but delete artifacts to save space. Criteria include number of builds, days old, or even custom scripts in pipelines to control retention.

Result

You understand flexible options to manage artifacts and builds separately.

Knowing you can keep build history while deleting artifacts helps balance traceability and storage.

5

Advanced

Retention policies in Jenkins pipelines

🤔Before reading on: do you think pipeline scripts can control artifact retention dynamically? Commit to your answer.

Concept: Learn how to implement artifact retention in Jenkins scripted or declarative pipelines.

In Jenkins pipelines, you use the 'buildDiscarder' step with 'logRotator' to set retention. For example: pipeline { options { buildDiscarder(logRotator(numToKeepStr: '10', daysToKeepStr: '30')) } stages { stage('Build') { steps { // build steps } } } } This dynamically applies retention rules as part of the pipeline code.

Result

You can automate artifact retention as part of pipeline scripts.

Integrating retention in pipelines enables consistent and code-driven storage management.

6

Advanced

Handling retention with external artifact storage

🤔Before reading on: do you think Jenkins retention policies affect artifacts stored outside Jenkins? Commit to your answer.

Concept: Understand how retention policies interact with artifacts stored in external systems like Nexus or Artifactory.

When Jenkins uploads artifacts to external repositories, retention policies inside Jenkins only affect local copies. External storage has its own retention rules. You must coordinate policies to avoid losing needed artifacts or wasting space.

Result

You know retention policies inside Jenkins do not control external artifact storage.

Recognizing this separation prevents accidental deletion and storage mismanagement.

7

Expert

Surprising retention pitfalls and solutions

🤔Before reading on: do you think deleting artifacts always frees disk space immediately? Commit to your answer.

Concept: Reveal subtle issues like disk space not freeing due to file locks or retention misconfigurations.

Sometimes Jenkins deletes artifacts but disk space doesn't free because files are still open by processes or plugins. Also, retention policies might not apply if jobs are misconfigured or artifacts are archived differently. Solutions include restarting Jenkins, checking plugins, and verifying retention settings carefully.

Result

You understand why artifact deletion might not immediately free space and how to fix it.

Knowing these pitfalls helps maintain Jenkins storage health and avoid confusing disk usage issues.

Under the Hood

Jenkins stores artifacts as files linked to build records on disk. Retention policies run periodically or after builds to check which builds exceed the configured limits. Jenkins then deletes artifact files but can keep build metadata. In pipelines, retention is enforced by steps that configure Jenkins' internal log rotator. External artifact repositories are separate systems with their own storage and retention mechanisms.

Why designed this way?

Jenkins separates build metadata from artifacts to allow flexible retention. Early Jenkins versions kept all artifacts, causing storage bloat. Introducing retention policies automated cleanup, reducing manual maintenance. The design balances traceability (keeping build info) and storage efficiency (deleting large artifacts). External repositories evolved to handle artifact storage at scale, so Jenkins delegates long-term storage to them.

┌───────────────┐       ┌───────────────┐
│ Build Record  │──────▶│ Artifact Files│
└──────┬────────┘       └──────┬────────┘
       │                       │
       │ Retention Policy Check │
       ▼                       ▼
┌───────────────┐       ┌───────────────┐
│ Keep or Delete│       │ Delete Files  │
└───────────────┘       └───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does Jenkins retention policy delete build records along with artifacts? Commit yes or no.

Common Belief: Retention policies delete both artifacts and build records together.

Reality: Retention policies can delete only artifacts while keeping build records intact.

Why it matters: Deleting build records unnecessarily loses build history and traceability.

Quick: Does Jenkins retention policy apply to artifacts stored in external repositories? Commit yes or no.

Common Belief: Jenkins retention policies control all artifact storage, including external repositories.

Reality: Jenkins retention policies only affect local artifact storage, not external repositories.

Why it matters: Assuming Jenkins deletes external artifacts can cause unexpected data loss or storage bloat.

Quick: Does deleting artifacts always immediately free disk space? Commit yes or no.

Common Belief: Deleting artifacts instantly frees disk space on the Jenkins server.

Reality: Sometimes disk space is not freed immediately due to file locks or system caching.

Why it matters: Misunderstanding this leads to confusion and unnecessary troubleshooting.

Quick: Can retention policies be configured globally for all Jenkins jobs? Commit yes or no.

Common Belief: Retention policies are set once globally and apply to all jobs automatically.

Reality: Retention policies are configured per job or pipeline; there is no global default.

Why it matters: Expecting global policies can cause inconsistent artifact management across projects.

Expert Zone

1

Retention policies can be combined with custom scripts in pipelines to implement complex rules based on build results or tags.

2

Some plugins or external tools may interfere with artifact deletion, requiring careful coordination and monitoring.

3

Retention settings impact not only storage but also backup strategies and compliance requirements.

When NOT to use

Avoid relying solely on Jenkins retention policies for long-term artifact storage; use dedicated artifact repositories like Nexus or Artifactory for scalable, reliable storage and retention management.

Production Patterns

In production, teams often use Jenkins retention policies to keep recent builds locally for quick access, while pushing artifacts to external repositories with their own retention. Pipelines include retention steps to automate cleanup, and monitoring alerts track disk usage to prevent outages.

Connections

Garbage Collection in Programming

Both automate removal of unused data to free resources.

Understanding artifact retention is like garbage collection: both clean up unused items to keep systems efficient.

Data Lifecycle Management in Storage Systems

Artifact retention policies are a form of data lifecycle management applied to build outputs.

Knowing data lifecycle concepts helps grasp how retention policies control data age and storage costs.

Inventory Management in Retail

Both involve deciding when to keep or discard items based on age and demand.

Retention policies are like managing store inventory: keep fresh items and remove old stock to optimize space and value.

Common Pitfalls

#1 Setting retention policy but forgetting to enable artifact archiving.

Wrong approach:

In Jenkins job:
// No 'Archive the artifacts' step configured
Discard old builds: keep 10 builds
// Artifacts never saved, so retention has no effect

Correct approach:

In Jenkins job:
Archive the artifacts: **/*.jar
Discard old builds: keep 10 builds
// Artifacts saved and retention deletes old ones

Root cause:

Retention policies only delete saved artifacts; if artifacts are not archived, retention settings do nothing.

#2 Expecting retention policy to delete artifacts stored externally.

Wrong approach:

Configure retention policy in Jenkins but rely on it to clean Nexus repository artifacts.

Correct approach:

Configure retention in Jenkins for local artifacts and set retention rules in Nexus for external storage.

Root cause:

Misunderstanding that Jenkins retention only affects local storage, not external repositories.

#3 Using 'days to keep' with zero or negative values causing no artifacts to be deleted.

Wrong approach:

Discard old builds: keep 10 builds, days to keep: 0
// No artifacts deleted because 0 means keep forever

Correct approach:

Discard old builds: keep 10 builds, days to keep: 30
// Artifacts older than 30 days deleted

Root cause:

Misinterpreting zero or negative days as immediate deletion instead of infinite retention.

Key Takeaways

Artifact retention policies automatically manage how long Jenkins keeps build files to save disk space and maintain system health.

Retention rules are configured per job or pipeline and can delete artifacts while preserving build history for traceability.

Jenkins retention policies only affect local artifact storage; external repositories require separate retention management.

Retention policies can be integrated into pipelines for automated, code-driven storage control.

Understanding retention pitfalls like file locks and misconfigurations helps avoid storage issues and keeps Jenkins running smoothly.

Practice

1. What is the main purpose of artifact retention policies in Jenkins?

easy

A. To manage disk space by deleting old build files

B. To speed up the build process by caching dependencies

C. To increase the number of builds stored indefinitely

D. To automatically trigger builds on code changes

Solution

1. Step 1: Understand artifact retention policies

   Artifact retention policies control how long build files are kept to avoid using too much disk space.

2. Step 2: Identify the main goal

   The main goal is to delete old build files to free up disk space.

3. Final Answer:

   To manage disk space by deleting old build files -> Option A

4. Quick Check:

   Artifact retention = disk space management [OK]

Hint: Retention policies delete old builds to save space [OK]

Common Mistakes:

• Confusing retention with build triggering
• Thinking retention speeds up builds
• Assuming retention keeps all builds forever

2. Which Jenkins pipeline syntax correctly sets a retention policy to keep only the last 5 builds?

easy

A. archiveArtifacts(numToKeepStr: '5')

B. buildDiscarder(keepLast: 5)

C. buildDiscarder(logRotator(numToKeepStr: '5'))

D. archiveArtifacts(logRotator(numToKeepStr: '5'))

Solution

1. Step 1: Identify correct syntax for retention

   Retention uses buildDiscarder with logRotator to specify how many builds to keep.

2. Step 2: Check option correctness

   buildDiscarder(logRotator(numToKeepStr: '5')) uses correct syntax: buildDiscarder(logRotator(numToKeepStr: '5')). Others misuse archiveArtifacts or wrong parameters.

3. Final Answer:

   buildDiscarder(logRotator(numToKeepStr: '5')) -> Option C

4. Quick Check:

   Retention uses buildDiscarder + logRotator [OK]

Hint: Retention = buildDiscarder with logRotator [OK]

Common Mistakes:

• Using archiveArtifacts to set retention
• Wrong parameter names like keepLast
• Nesting logRotator inside archiveArtifacts

3. Given this Jenkins pipeline snippet:

pipeline {
  options {
    buildDiscarder(logRotator(numToKeepStr: '3', daysToKeepStr: '7'))
  }
  stages {
    stage('Build') {
      steps {
        echo 'Building...'
      }
    }
  }
}

What happens to builds older than 7 days or beyond the last 3 builds?

medium

A. They are deleted to free disk space

B. They are archived permanently

C. They trigger a new build automatically

D. They are kept indefinitely

Solution

1. Step 1: Understand logRotator parameters

   It keeps only the last 3 builds and deletes builds older than 7 days.

2. Step 2: Determine effect on old builds

   Builds older than 7 days or beyond the last 3 are deleted to save space.

3. Final Answer:

   They are deleted to free disk space -> Option A

4. Quick Check:

   logRotator deletes old builds [OK]

Hint: logRotator deletes builds older than days or count [OK]

Common Mistakes:

• Thinking builds are archived permanently
• Assuming builds trigger new builds
• Believing builds are kept forever

4. You wrote this Jenkins pipeline snippet to keep artifacts for 10 days:

options {
  buildDiscarder(logRotator(artifactDaysToKeep: '10'))
}

But artifacts are not deleted after 10 days. What is the likely problem?

medium

A. logRotator does not support artifact retention

B. The buildDiscarder option must be inside stages

C. You must use archiveArtifacts to delete old artifacts

D. The parameter name should be artifactDaysToKeepStr, not artifactDaysToKeep

Solution

1. Step 1: Check logRotator parameter names

   logRotator parameters for strings end with 'Str', like artifactDaysToKeepStr.

2. Step 2: Identify the syntax issue

   Missing 'Str' means the parameter is not recognized, so default retention applies and artifacts aren't deleted after 10 days.

3. Final Answer:

   The parameter name should be artifactDaysToKeepStr, not artifactDaysToKeep -> Option D

4. Quick Check:

   logRotator params end with 'Str' [OK]

Hint: logRotator params like artifactDaysToKeepStr end with Str [OK]

Common Mistakes:

• Using wrong parameter names
• Expecting logRotator to delete artifacts
• Placing buildDiscarder outside options block

5. You want to keep artifacts only for the last 4 builds and delete older artifacts automatically, while keeping older builds. Which Jenkins pipeline snippet correctly applies this retention policy?

hard

A. steps { archiveArtifacts artifacts: '**/*', onlyIfSuccessful: true, retentionDays: 4 }

B. options { buildDiscarder(logRotator(artifactNumToKeepStr: '4')) }

C. options { buildDiscarder(logRotator(numToKeepStr: '4')) } steps { archiveArtifacts artifacts: '**/*', onlyIfSuccessful: true }

D. steps { archiveArtifacts artifacts: '**/*', keepOnlyLast: 4 }

Solution

1. Step 1: Understand roles for artifact-specific retention

   To keep artifacts from only the last 4 builds while retaining builds, use artifactNumToKeepStr.

2. Step 2: Check option correctness

   options { buildDiscarder(logRotator(artifactNumToKeepStr: '4')) }: buildDiscarder(logRotator(artifactNumToKeepStr: '4')) keeps artifacts from last 4 builds, deletes older artifacts, keeps all builds. C deletes old builds too. B/D use invalid params.

3. Final Answer:

   options { buildDiscarder(logRotator(artifactNumToKeepStr: '4')) } -> Option B

4. Quick Check:

   artifactNumToKeepStr for artifacts only [OK]

Hint: artifactNumToKeepStr keeps artifacts from last N builds [OK]

Common Mistakes:

• Using invalid parameters like retentionDays or keepOnlyLast
• Trying to control retention only via archiveArtifacts
• Misplacing buildDiscarder outside options block