Artifact retention policies in Jenkins - Deep Dive

Overview - Artifact retention policies

What is it?

Artifact retention policies are rules that decide how long build outputs, called artifacts, are kept in Jenkins. Artifacts are files like compiled programs or reports created during a build. These policies help manage storage by automatically deleting old or unnecessary artifacts. This keeps the Jenkins environment clean and efficient.

Why it matters

Without artifact retention policies, Jenkins would store every artifact forever, quickly using up disk space and slowing down the system. This can cause failures or delays in builds and increase maintenance work. Proper retention policies save storage, reduce costs, and keep the build system reliable and fast.

Where it fits

Before learning artifact retention policies, you should understand Jenkins basics like jobs, builds, and artifacts. After this, you can explore advanced Jenkins maintenance, storage optimization, and pipeline automation techniques.

Mental Model

Core Idea

Artifact retention policies are automated rules that decide when to keep or delete build files to save space and keep Jenkins healthy.

Think of it like...

It's like cleaning out your fridge regularly: you keep fresh food you need and throw away old leftovers to make room for new groceries.

┌───────────────────────────────┐
│         Jenkins Build          │
│  ┌───────────────┐            │
│  │  Artifacts    │            │
│  └──────┬────────┘            │
│         │                    │
│  ┌──────▼────────┐           │
│  │Retention Rules│           │
│  └──────┬────────┘           │
│         │                    │
│  ┌──────▼────────┐           │
│  │ Keep or Delete│           │
│  └───────────────┘           │
└───────────────────────────────┘

Build-Up - 7 Steps

1

Foundation

What are Jenkins artifacts

Concept: Introduce what artifacts are in Jenkins and why they matter.

In Jenkins, an artifact is any file or set of files produced by a build job. For example, a compiled program, a test report, or a zipped package. Artifacts are saved so you can use them later, like deploying or sharing results.

Result

You understand that artifacts are build outputs stored by Jenkins for future use.

Knowing what artifacts are helps you see why managing their storage is important.

2

Foundation

Why artifact retention is needed

Concept: Explain the problem of unlimited artifact storage.

Every build can create artifacts, and if Jenkins keeps all of them forever, disk space fills up fast. This slows down Jenkins and can cause failures. Retention policies prevent this by deleting old or unneeded artifacts automatically.

Result

You see the need for rules to control how long artifacts stay in Jenkins.

Understanding the storage problem motivates the use of retention policies.

3

Intermediate

Configuring retention policies in Jenkins

🤔Before reading on: do you think retention policies are set globally or per job? Commit to your answer.

Concept: Learn how to set artifact retention rules in Jenkins job configurations.

In Jenkins, you configure retention policies inside each job's settings under 'Discard old builds'. You can specify how many builds or days to keep. For example, keep artifacts for the last 10 builds or 30 days. Jenkins then deletes artifacts from older builds automatically.

Result

You can set rules to keep or delete artifacts based on build count or age.

Knowing that retention is per job lets you customize policies for different projects.

4

Intermediate

Retention policy options and criteria

🤔Before reading on: do you think Jenkins can delete artifacts but keep build records? Commit to your answer.

Concept: Explore different criteria Jenkins uses to decide artifact deletion.

Jenkins allows you to discard builds entirely or just their artifacts. You can keep build records but delete artifacts to save space. Criteria include number of builds, days old, or even custom scripts in pipelines to control retention.

Result

You understand flexible options to manage artifacts and builds separately.

Knowing you can keep build history while deleting artifacts helps balance traceability and storage.

5

Advanced

Retention policies in Jenkins pipelines

🤔Before reading on: do you think pipeline scripts can control artifact retention dynamically? Commit to your answer.

Concept: Learn how to implement artifact retention in Jenkins scripted or declarative pipelines.

In Jenkins pipelines, you use the 'buildDiscarder' step with 'logRotator' to set retention. For example: pipeline { options { buildDiscarder(logRotator(numToKeepStr: '10', daysToKeepStr: '30')) } stages { stage('Build') { steps { // build steps } } } } This dynamically applies retention rules as part of the pipeline code.

Result

You can automate artifact retention as part of pipeline scripts.

Integrating retention in pipelines enables consistent and code-driven storage management.

6

Advanced

Handling retention with external artifact storage

🤔Before reading on: do you think Jenkins retention policies affect artifacts stored outside Jenkins? Commit to your answer.

Concept: Understand how retention policies interact with artifacts stored in external systems like Nexus or Artifactory.

When Jenkins uploads artifacts to external repositories, retention policies inside Jenkins only affect local copies. External storage has its own retention rules. You must coordinate policies to avoid losing needed artifacts or wasting space.

Result

You know retention policies inside Jenkins do not control external artifact storage.

Recognizing this separation prevents accidental deletion and storage mismanagement.

7

Expert

Surprising retention pitfalls and solutions

🤔Before reading on: do you think deleting artifacts always frees disk space immediately? Commit to your answer.

Concept: Reveal subtle issues like disk space not freeing due to file locks or retention misconfigurations.

Sometimes Jenkins deletes artifacts but disk space doesn't free because files are still open by processes or plugins. Also, retention policies might not apply if jobs are misconfigured or artifacts are archived differently. Solutions include restarting Jenkins, checking plugins, and verifying retention settings carefully.

Result

You understand why artifact deletion might not immediately free space and how to fix it.

Knowing these pitfalls helps maintain Jenkins storage health and avoid confusing disk usage issues.

Under the Hood

Jenkins stores artifacts as files linked to build records on disk. Retention policies run periodically or after builds to check which builds exceed the configured limits. Jenkins then deletes artifact files but can keep build metadata. In pipelines, retention is enforced by steps that configure Jenkins' internal log rotator. External artifact repositories are separate systems with their own storage and retention mechanisms.

Why designed this way?

Jenkins separates build metadata from artifacts to allow flexible retention. Early Jenkins versions kept all artifacts, causing storage bloat. Introducing retention policies automated cleanup, reducing manual maintenance. The design balances traceability (keeping build info) and storage efficiency (deleting large artifacts). External repositories evolved to handle artifact storage at scale, so Jenkins delegates long-term storage to them.

┌───────────────┐       ┌───────────────┐
│ Build Record  │──────▶│ Artifact Files│
└──────┬────────┘       └──────┬────────┘
       │                       │
       │ Retention Policy Check │
       ▼                       ▼
┌───────────────┐       ┌───────────────┐
│ Keep or Delete│       │ Delete Files  │
└───────────────┘       └───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does Jenkins retention policy delete build records along with artifacts? Commit yes or no.

Common Belief: Retention policies delete both artifacts and build records together.

Reality: Retention policies can delete only artifacts while keeping build records intact.

Why it matters: Deleting build records unnecessarily loses build history and traceability.

Quick: Does Jenkins retention policy apply to artifacts stored in external repositories? Commit yes or no.

Common Belief: Jenkins retention policies control all artifact storage, including external repositories.

Reality: Jenkins retention policies only affect local artifact storage, not external repositories.

Why it matters: Assuming Jenkins deletes external artifacts can cause unexpected data loss or storage bloat.

Quick: Does deleting artifacts always immediately free disk space? Commit yes or no.

Common Belief: Deleting artifacts instantly frees disk space on the Jenkins server.

Reality: Sometimes disk space is not freed immediately due to file locks or system caching.

Why it matters: Misunderstanding this leads to confusion and unnecessary troubleshooting.

Quick: Can retention policies be configured globally for all Jenkins jobs? Commit yes or no.

Common Belief: Retention policies are set once globally and apply to all jobs automatically.

Reality: Retention policies are configured per job or pipeline; there is no global default.

Why it matters: Expecting global policies can cause inconsistent artifact management across projects.

Expert Zone

1

Retention policies can be combined with custom scripts in pipelines to implement complex rules based on build results or tags.

2

Some plugins or external tools may interfere with artifact deletion, requiring careful coordination and monitoring.

3

Retention settings impact not only storage but also backup strategies and compliance requirements.

When NOT to use

Avoid relying solely on Jenkins retention policies for long-term artifact storage; use dedicated artifact repositories like Nexus or Artifactory for scalable, reliable storage and retention management.

Production Patterns

In production, teams often use Jenkins retention policies to keep recent builds locally for quick access, while pushing artifacts to external repositories with their own retention. Pipelines include retention steps to automate cleanup, and monitoring alerts track disk usage to prevent outages.

Connections

Garbage Collection in Programming

Both automate removal of unused data to free resources.

Understanding artifact retention is like garbage collection: both clean up unused items to keep systems efficient.

Data Lifecycle Management in Storage Systems

Artifact retention policies are a form of data lifecycle management applied to build outputs.

Knowing data lifecycle concepts helps grasp how retention policies control data age and storage costs.

Inventory Management in Retail

Both involve deciding when to keep or discard items based on age and demand.

Retention policies are like managing store inventory: keep fresh items and remove old stock to optimize space and value.

Common Pitfalls

#1 Setting retention policy but forgetting to enable artifact archiving.

Wrong approach:

In Jenkins job:
// No 'Archive the artifacts' step configured
Discard old builds: keep 10 builds
// Artifacts never saved, so retention has no effect

Correct approach:

In Jenkins job:
Archive the artifacts: **/*.jar
Discard old builds: keep 10 builds
// Artifacts saved and retention deletes old ones

Root cause:

Retention policies only delete saved artifacts; if artifacts are not archived, retention settings do nothing.

#2 Expecting retention policy to delete artifacts stored externally.

Wrong approach:

Configure retention policy in Jenkins but rely on it to clean Nexus repository artifacts.

Correct approach:

Configure retention in Jenkins for local artifacts and set retention rules in Nexus for external storage.

Root cause:

Misunderstanding that Jenkins retention only affects local storage, not external repositories.

#3 Using 'days to keep' with zero or negative values causing no artifacts to be deleted.

Wrong approach:

Discard old builds: keep 10 builds, days to keep: 0
// No artifacts deleted because 0 means keep forever

Correct approach:

Discard old builds: keep 10 builds, days to keep: 30
// Artifacts older than 30 days deleted

Root cause:

Misinterpreting zero or negative days as immediate deletion instead of infinite retention.

Key Takeaways

Artifact retention policies automatically manage how long Jenkins keeps build files to save disk space and maintain system health.

Retention rules are configured per job or pipeline and can delete artifacts while preserving build history for traceability.

Jenkins retention policies only affect local artifact storage; external repositories require separate retention management.

Retention policies can be integrated into pipelines for automated, code-driven storage control.

Understanding retention pitfalls like file locks and misconfigurations helps avoid storage issues and keeps Jenkins running smoothly.