Why shared libraries reduce duplication in Jenkins - Why It Works This Way

Overview - Why shared libraries reduce duplication

What is it?

Shared libraries in Jenkins are reusable collections of code and scripts that multiple Jenkins pipelines can use. They help avoid rewriting the same code in different pipeline jobs by centralizing common functions and steps. This makes pipeline management easier and more consistent across projects. Essentially, they act like a toolbox that all pipelines can access.

Why it matters

Without shared libraries, teams often copy and paste the same pipeline code in many places. This leads to errors, inconsistent behavior, and extra work when updating pipelines. Shared libraries solve this by letting teams write code once and use it everywhere, saving time and reducing mistakes. This improves reliability and speeds up delivery.

Where it fits

Before learning about shared libraries, you should understand basic Jenkins pipelines and how to write simple scripted or declarative pipelines. After mastering shared libraries, you can explore advanced pipeline design, pipeline as code best practices, and Jenkins plugin development.

Mental Model

Core Idea

Shared libraries let you write pipeline code once and reuse it everywhere, preventing repeated work and mistakes.

Think of it like...

Using shared libraries is like having a shared recipe book in a kitchen where everyone cooks. Instead of each cook writing their own recipe from scratch, they all use the same trusted recipes, ensuring consistent dishes and saving time.

┌─────────────────────┐
│  Jenkins Pipelines  │
├─────────┬───────────┤
│ Pipeline│ Pipeline  │
│   A     │    B      │
└────┬────┴────┬──────┘
     │         │
     ▼         ▼
┌─────────────────────┐
│  Shared Library      │
│  (Reusable Code)     │
└─────────────────────┘

Build-Up - 6 Steps

1

Foundation

What is a Jenkins shared library

Concept: Introduce the idea of a shared library as a reusable code collection for Jenkins pipelines.

A Jenkins shared library is a set of Groovy scripts and functions stored in a separate repository. Pipelines can load this library to use its code. This avoids copying the same code into every pipeline. Instead, pipelines call shared functions from the library.

Result

Pipelines can call shared functions instead of duplicating code.

Understanding that shared libraries are separate code bundles helps see how reuse is possible across many pipelines.

2

Foundation

How duplication happens in pipelines

Concept: Explain why pipeline code often gets copied and pasted.

When teams write pipelines, they often need the same steps like building, testing, or deploying. Without shared libraries, they copy these steps into each pipeline. This causes duplication because the same code exists in many places.

Result

Multiple pipelines have repeated code blocks doing the same tasks.

Recognizing duplication as repeated code helps motivate the need for a shared solution.

3

Intermediate

Using shared libraries to centralize code

🤔Before reading on: do you think using shared libraries means pipelines must be rewritten completely or just call shared functions? Commit to your answer.

Concept: Show how pipelines can call shared library functions instead of copying code.

Instead of copying code, pipelines load the shared library and call its functions. For example, a 'build' function in the library can be called by many pipelines. This centralizes code in one place.

Result

Pipelines become shorter and simpler, calling shared functions.

Knowing pipelines can call shared code without rewriting everything shows how duplication is reduced.

4

Intermediate

Updating shared code updates all pipelines

🤔Before reading on: if you fix a bug in a shared library, does it fix the bug in all pipelines immediately or only new pipelines? Commit to your answer.

Concept: Explain that changing shared library code affects all pipelines using it.

When you update a function in the shared library, all pipelines that use it get the update next time they run. This means fixes and improvements apply everywhere at once.

Result

Bug fixes and improvements propagate to all pipelines automatically.

Understanding this shows how shared libraries improve maintainability and consistency.

5

Advanced

Structuring shared libraries for scalability

🤔Before reading on: do you think a shared library should contain all code in one file or be organized into multiple files and folders? Commit to your answer.

Concept: Teach how to organize shared libraries into folders and multiple scripts for clarity and reuse.

Shared libraries can have multiple Groovy files organized by function, like 'build.groovy', 'deploy.groovy', and helper scripts. This modular structure helps teams find and update code easily.

Result

Shared libraries become easier to maintain and scale as projects grow.

Knowing how to organize shared libraries prevents chaos and supports team collaboration.

6

Expert

Versioning and compatibility in shared libraries

🤔Before reading on: do you think all pipelines should always use the latest shared library version or can they use specific versions? Commit to your answer.

Concept: Explain how to manage shared library versions to avoid breaking pipelines.

Jenkins supports versioning shared libraries using Git tags or branches. Pipelines can specify which version to use. This prevents breaking pipelines when the library changes and allows gradual upgrades.

Result

Teams can safely update shared libraries without disrupting running pipelines.

Understanding version control in shared libraries is key to managing risk in production environments.

Under the Hood

Jenkins loads shared libraries from a Git repository configured in Jenkins settings. When a pipeline runs, Jenkins fetches the library code and makes its functions available as Groovy methods. Pipelines call these methods as if they were local functions. The library code runs in the same JVM as the pipeline, sharing context and variables.

Why designed this way?

Shared libraries were designed to separate reusable pipeline code from individual pipelines to avoid duplication and ease maintenance. Using Git for storage leverages existing version control tools. Loading code dynamically allows pipelines to stay lightweight and flexible.

┌───────────────┐        ┌───────────────┐
│ Jenkins Job   │───────▶│ Shared Library │
│ (Pipeline)    │        │ (Git Repo)    │
└───────────────┘        └───────────────┘
        │                        ▲
        │ Calls functions        │ Code fetched at runtime
        ▼                        │
┌───────────────────────────────┐
│ Jenkins Pipeline Execution JVM │
│ - Runs pipeline and library   │
└───────────────────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does using shared libraries mean you never write any pipeline code in the Jenkinsfile? Commit yes or no.

Common Belief: Some think shared libraries replace all pipeline code, so Jenkinsfiles become empty.

Reality: Jenkinsfiles still define pipeline structure and call shared library functions; they don't disappear.

Why it matters: Expecting empty Jenkinsfiles leads to confusion and misuse of shared libraries.

Quick: Do you think updating a shared library always immediately fixes all pipelines without any testing? Commit yes or no.

Common Belief: People believe shared library updates instantly fix all pipelines safely.

Reality: Updates can break pipelines if not tested; versioning and careful rollout are needed.

Why it matters: Ignoring testing risks pipeline failures and downtime.

Quick: Is it true that shared libraries automatically improve pipeline performance? Commit yes or no.

Common Belief: Some assume shared libraries make pipelines run faster by default.

Reality: Shared libraries reduce duplication but don't inherently speed up execution.

Why it matters: Misunderstanding this can lead to wrong expectations and ignoring real performance tuning.

Quick: Do you think shared libraries are only useful for large teams? Commit yes or no.

Common Belief: Many believe shared libraries are only for big organizations.

Reality: Even small teams benefit by reducing errors and saving time.

Why it matters: Small teams may miss out on efficiency gains by avoiding shared libraries.

Expert Zone

1

Shared libraries can include global variables and custom steps that integrate deeply with Jenkins internals, enabling powerful pipeline abstractions.

2

Managing library dependencies and avoiding circular references requires careful design to keep libraries maintainable.

3

Using shared libraries with multibranch pipelines requires understanding how library versions and branches interact to avoid unexpected behavior.

When NOT to use

Avoid shared libraries for very simple pipelines or one-off jobs where the overhead of maintaining a library outweighs benefits. Alternatives include inline pipeline scripts or lightweight templates.

Production Patterns

In production, teams use shared libraries to enforce company-wide standards, implement common deployment steps, and provide utility functions. They often combine libraries with version control and automated testing to ensure safe pipeline updates.

Connections

Software Libraries in Programming

Shared libraries in Jenkins are similar to software libraries in programming languages that provide reusable code modules.

Understanding software libraries helps grasp how Jenkins shared libraries promote code reuse and modularity.

Version Control Systems

Shared libraries rely on Git for versioning and distribution, connecting pipeline code reuse with version control best practices.

Knowing Git concepts like branches and tags helps manage shared library versions safely.

Manufacturing Assembly Lines

Just like assembly lines reuse standard parts to build products efficiently, shared libraries reuse code to build pipelines efficiently.

Seeing pipelines as assembly lines clarifies why reusing tested parts reduces errors and speeds up delivery.

Common Pitfalls

#1 Copying shared library code into each pipeline instead of calling it.

Wrong approach:

pipeline {
stages {
stage('Build') {
steps {
sh 'mvn clean install'
}
}
}
}
// duplicated in every pipeline

Correct approach:

@Library('my-shared-lib') _
pipeline {
stages {
stage('Build') {
steps {
buildProject()
}
}
}
}
// calls shared library function

Root cause:

Not understanding how to reference shared library functions leads to duplication.

#2 Updating shared library code without version control or testing.

Wrong approach:

Directly pushing changes to the shared library master branch without tests.

Correct approach:

Create a feature branch, test changes, then merge with code review and tagging.

Root cause:

Ignoring versioning and testing risks breaking pipelines.

#3 Assuming shared libraries improve pipeline speed automatically.

Wrong approach:

Relying on shared libraries to fix slow builds without profiling or optimization.

Correct approach:

Use shared libraries for code reuse, and separately optimize pipeline performance.

Root cause:

Confusing code reuse with execution speed.

Key Takeaways

Shared libraries in Jenkins centralize reusable pipeline code, preventing duplication across multiple pipelines.

Using shared libraries makes pipelines simpler, easier to maintain, and consistent across projects.

Updating shared library code updates all pipelines that use it, improving maintainability but requiring careful version control.

Proper organization and versioning of shared libraries are essential for scalable and safe pipeline management.

Understanding shared libraries connects Jenkins pipeline development with broader software engineering practices like modularity and version control.