Stash and unstash for passing data in Jenkins - Deep Dive

Overview - Stash and unstash for passing data

What is it?

Stash and unstash are Jenkins pipeline steps used to temporarily save files during a build and retrieve them later in the same pipeline. Stash saves a set of files with a name, and unstash restores those files by that name. This helps pass data between different stages or nodes in a Jenkins pipeline without using external storage.

Why it matters

Without stash and unstash, sharing files between pipeline stages or nodes would require complex workarounds like writing to external storage or duplicating work. This would slow down builds and increase errors. Stash and unstash make pipelines simpler, faster, and more reliable by managing temporary data efficiently.

Where it fits

Before learning stash and unstash, you should understand Jenkins pipelines, stages, and workspace basics. After mastering stash and unstash, you can explore advanced pipeline features like parallel stages, distributed builds, and artifact archiving.

Mental Model

Core Idea

Stash saves files temporarily in Jenkins workspace so they can be retrieved later in the pipeline by unstash, enabling data sharing across stages or nodes.

Think of it like...

Imagine you pack your lunch in a labeled box (stash) before leaving home, then pick up that box later at work (unstash) to eat. The box keeps your food safe and ready, just like stash keeps files safe between pipeline steps.

Pipeline Flow:

[Stage 1: Build] ──stash──▶ [Temporary Storage]

[Stage 2: Test] ◀─unstash── [Temporary Storage]

[Stage 3: Deploy] ◀─unstash── [Temporary Storage]

Each arrow shows saving or retrieving files by name.

Build-Up - 7 Steps

1

Foundation

Understanding Jenkins Workspace Basics

Concept: Learn what the Jenkins workspace is and how files are stored during a build.

The Jenkins workspace is a folder on the build agent where your pipeline runs. All files created or checked out during the build live here. Each stage runs in this workspace unless you use different nodes.

Result

You know where files live during a build and that stages share the same workspace by default on one node.

Understanding the workspace is key because stash and unstash move files between workspaces or nodes, which are otherwise isolated.

2

Foundation

Why Sharing Files Between Stages Is Hard

Concept: Recognize the problem of passing files between stages, especially on different nodes.

If your pipeline runs stages on different machines (nodes), each has its own workspace. Files created on one node are not automatically available on another. This breaks workflows that need files from earlier stages.

Result

You see why simply creating files in one stage doesn't make them available in another on a different node.

Knowing this problem sets the stage for why stash and unstash are needed to move files safely.

3

Intermediate

Using stash to Save Files Temporarily

🤔Before reading on: do you think stash copies files to a permanent storage or just keeps them temporarily? Commit to your answer.

Concept: Learn how to use the stash step to save files by name during a pipeline.

In a pipeline stage, you use stash with a name and a file pattern. For example: stash(name: 'myFiles', includes: '**/*.jar') This saves all .jar files in the workspace under the name 'myFiles'.

Result

Files matching the pattern are saved temporarily and can be retrieved later by unstash.

Understanding stash as a temporary, named snapshot of files helps you organize data flow in pipelines.

4

Intermediate

Using unstash to Retrieve Saved Files

🤔Before reading on: do you think unstash deletes files after retrieving or keeps them for reuse? Commit to your answer.

Concept: Learn how to use unstash to restore files saved by stash in later stages.

In a later stage, you call unstash with the same name: unstash 'myFiles' This copies the saved files back into the current workspace, making them available for use.

Result

Files saved earlier are restored exactly as they were, ready for the next steps.

Knowing unstash restores files by name allows you to pass data reliably between disconnected pipeline parts.

5

Intermediate

Stash and Unstash Across Different Nodes

Concept: See how stash and unstash enable file sharing when stages run on different machines.

When a stage runs on one node and stashes files, Jenkins stores them centrally. Later, a stage on another node can unstash those files, copying them into its workspace. This overcomes workspace isolation.

Result

Files flow seamlessly between nodes without manual copying or external storage.

Understanding this cross-node transfer is crucial for building scalable, distributed pipelines.

6

Advanced

Limitations and Performance of Stash/Unstash

🤔Before reading on: do you think stash stores files uncompressed or compressed? Commit to your answer.

Concept: Explore how stash compresses files and the impact on large data or many files.

Jenkins compresses stashed files into a zip archive for transfer and storage. Large files or many small files can slow down stash/unstash steps. Also, stash storage is temporary and limited by Jenkins master disk space.

Result

You understand stash is efficient but has practical limits and can affect build speed.

Knowing stash compresses files explains why it is fast but why very large data should be handled differently.

7

Expert

Advanced Patterns: Combining Stash with Parallel and Matrix Builds

🤔Before reading on: do you think stash names must be unique across parallel branches? Commit to your answer.

Concept: Learn how to use stash and unstash in complex pipelines with parallel or matrix stages.

In parallel branches, stash names must be unique per branch to avoid conflicts. You can use dynamic names like stash(name: "myFiles-${env.BRANCH_NAME}", includes: '**/*.jar'). Also, unstash only restores files saved in the same pipeline run, so coordination is key. Example: parallel { branch1 { stash(name: 'files-branch1', includes: '**/*.jar') } branch2 { stash(name: 'files-branch2', includes: '**/*.jar') } } Later: unstash 'files-branch1' unstash 'files-branch2'

Result

You can safely share files in complex pipelines without overwriting or losing data.

Understanding stash naming and scope prevents subtle bugs in parallel pipelines.

Under the Hood

When you call stash, Jenkins compresses the specified files into a zip archive and stores it temporarily on the Jenkins master server. This archive is indexed by the stash name and pipeline run. When unstash is called, Jenkins retrieves the archive, decompresses it, and copies the files into the current workspace on the agent node. This process allows files to move between isolated workspaces and nodes transparently.

Why designed this way?

Jenkins pipelines often run on multiple agents with isolated workspaces. Direct file sharing is impossible without a central store. Stash/unstash were designed to provide a simple, built-in way to move files without requiring external storage or complex scripting. Compression reduces network load and storage use. The design balances ease of use, performance, and reliability.

┌───────────────┐        stash        ┌───────────────┐
│  Agent Node 1 │ ────────────────▶ │ Jenkins Master│
│ Workspace A   │                   │ (Temp Storage)│
└───────────────┘                   └───────────────┘
         ▲                                  │
         │                                  │ unstash
         │                                  ▼
┌───────────────┐                   ┌───────────────┐
│  Agent Node 2 │ ◀─────────────── │ Jenkins Master│
│ Workspace B   │                   │ (Temp Storage)│
└───────────────┘                   └───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does unstash automatically delete the stashed files after restoring? Commit yes or no.

Common Belief: Unstash removes the stashed files from Jenkins storage after restoring them.

Reality: Unstash copies the files but does not delete the stash archive; the stash remains available until the pipeline ends or Jenkins cleans it up.

Why it matters: Assuming unstash deletes stash can cause confusion when trying to unstash multiple times or in parallel stages.

Quick: Can stash share files between different pipeline runs? Commit yes or no.

Common Belief: Stash archives can be used to share files between separate pipeline runs or jobs.

Reality: Stash archives are scoped to a single pipeline run and cannot be shared across different runs or jobs.

Why it matters: Trying to unstash from a different run will fail, leading to build errors and wasted debugging time.

Quick: Does stash store files permanently on Jenkins master? Commit yes or no.

Common Belief: Stash saves files permanently on the Jenkins master for future use.

Reality: Stash stores files temporarily only for the duration of the pipeline run; files are deleted afterward.

Why it matters: Expecting permanent storage can cause data loss if you rely on stash for long-term artifact storage.

Quick: Does stash transfer files uncompressed? Commit yes or no.

Common Belief: Stash transfers files as-is without compression.

Reality: Stash compresses files into a zip archive before transfer to save bandwidth and storage.

Why it matters: Not knowing this can lead to misunderstandings about stash performance and size limits.

Expert Zone

1

Stash archives are stored on the Jenkins master and can cause disk space issues if large or numerous stashes accumulate during many concurrent builds.

2

Using stash with many small files can be slower than bundling files into a single archive file before stashing, due to overhead in compression and transfer.

3

Stash names must be unique within a pipeline run; reusing names in parallel branches without differentiation causes conflicts and unpredictable behavior.

When NOT to use

Avoid stash/unstash for very large files or artifacts that need to persist beyond the pipeline run. Instead, use artifact repositories like Nexus or Artifactory, or Jenkins' archiveArtifacts step for long-term storage and sharing.

Production Patterns

In production, stash/unstash is commonly used to pass build outputs like compiled binaries or test reports between build, test, and deploy stages. Teams often combine stash with parallel stages, using dynamic stash names to isolate data per branch or environment. Monitoring stash size and cleaning up old builds prevents master disk overload.

Connections

Artifact Repositories

stash/unstash provide temporary file sharing, while artifact repositories store build outputs permanently.

Understanding stash limits helps you decide when to use temporary sharing versus permanent artifact storage.

Distributed Systems

stash/unstash solve the problem of sharing state (files) across isolated nodes in a distributed build system.

Recognizing stash as a state-sharing mechanism clarifies challenges in distributed computing.

Package Delivery Logistics

stash is like a temporary warehouse holding packages before delivery to different destinations (pipeline stages).

This connection highlights the importance of temporary storage and naming for efficient delivery workflows.

Common Pitfalls

#1 Using the same stash name in parallel branches causing file conflicts.

Wrong approach:

parallel {
branch1 {
stash(name: 'sharedFiles', includes: '**/*.jar')
}
branch2 {
stash(name: 'sharedFiles', includes: '**/*.jar')
}
}

Correct approach:

parallel {
branch1 {
stash(name: 'sharedFiles-branch1', includes: '**/*.jar')
}
branch2 {
stash(name: 'sharedFiles-branch2', includes: '**/*.jar')
}
}

Root cause:

Not realizing stash names must be unique per pipeline run to avoid overwriting archives.

#2 Trying to unstash files before they are stashed in the pipeline.

Wrong approach:

stage('Test') {
unstash 'buildFiles'
}
stage('Build') {
stash(name: 'buildFiles', includes: '**/*.jar')
}

Correct approach:

stage('Build') {
stash(name: 'buildFiles', includes: '**/*.jar')
}
stage('Test') {
unstash 'buildFiles'
}

Root cause:

Misunderstanding pipeline execution order causes unstash to fail because stash does not exist yet.

#3 Using stash to store very large files causing slow builds and disk issues.

Wrong approach:

stash(name: 'largeData', includes: '**/hugefile.iso')

Correct approach:

Use external artifact storage or archiveArtifacts step for large files instead of stash.

Root cause:

Not knowing stash is intended for temporary, relatively small file sets.

Key Takeaways

Stash and unstash let Jenkins pipelines share files temporarily between stages and nodes by saving and restoring named file sets.

They solve the problem of isolated workspaces in distributed builds without needing external storage.

Stash compresses files and stores them temporarily on the Jenkins master, so stash names must be unique and files are not permanent.

Using stash correctly in parallel or matrix pipelines requires careful naming to avoid conflicts.

For large or permanent artifacts, use dedicated artifact repositories or Jenkins archiving instead of stash.