Options directive (timeout, retry) in Jenkins - Deep Dive

Overview - Options directive (timeout, retry)

What is it?

The Options directive in Jenkins Pipeline allows you to set special behaviors for your pipeline or stages, such as limiting how long a step can run or how many times it should retry on failure. Two common options are timeout, which stops a step if it runs too long, and retry, which repeats a step if it fails. These options help make your automation more reliable and efficient by handling delays and errors automatically.

Why it matters

Without timeout and retry options, pipelines can hang forever or fail without trying again, causing delays and wasted resources. These options prevent stuck builds and improve success rates by managing time limits and automatic retries. This means faster feedback and more stable automation, which is crucial for teams relying on continuous integration and delivery.

Where it fits

Before learning Options directive, you should understand basic Jenkins Pipelines and how to write stages and steps. After mastering options like timeout and retry, you can explore advanced error handling, parallel execution, and custom pipeline libraries to build robust automation workflows.

Mental Model

Core Idea

Options directive controls how long a step runs and how many times it retries to keep pipelines efficient and resilient.

Think of it like...

It's like setting a timer and retry button on a microwave: if the food takes too long, the timer stops it, and if it didn't cook well, you can press retry to try again without starting over.

Pipeline
  ├─ Stage 1
  │    ├─ Step A (timeout: 10 mins, retry: 3)
  │    └─ Step B
  └─ Stage 2
       └─ Step C (timeout: 5 mins)

Options directive applies rules like:
  ├─ timeout: stops step if it exceeds time
  └─ retry: repeats step on failure

Build-Up - 7 Steps

1

Foundation

Understanding Jenkins Pipeline Basics

Concept: Learn what a Jenkins Pipeline is and how it runs steps and stages.

A Jenkins Pipeline is a script that defines a series of steps to automate tasks like building or testing code. It has stages to organize work and steps inside those stages to do specific actions. For example, a stage might compile code, and a step inside it runs the compiler command.

Result

You can write a simple pipeline that runs commands in order.

Knowing the basic structure of pipelines is essential before adding controls like timeout or retry.

2

Foundation

What Are Pipeline Options?

Concept: Options are special settings that change how the pipeline or stages behave.

Options let you add rules like how long a step can run or how many times to retry on failure. They are declared inside the pipeline or stage block using the options directive. For example, options { timeout(time: 5, unit: 'MINUTES') } stops a step after 5 minutes.

Result

You can add options to control pipeline behavior.

Options give you control over pipeline execution beyond just running commands.

3

Intermediate

Using Timeout Option to Limit Step Duration

🤔Before reading on: do you think timeout stops the entire pipeline or just the step it's applied to? Commit to your answer.

Concept: Timeout stops a step or stage if it runs longer than the specified time.

The timeout option takes parameters like time and unit (SECONDS, MINUTES, HOURS). When the time limit is reached, Jenkins aborts the step or stage and marks it as failed. Example: pipeline { agent any options { timeout(time: 10, unit: 'MINUTES') } stages { stage('Build') { steps { sh 'sleep 600' // simulate long task } } } } This stops the build if it runs longer than 10 minutes.

Result

Steps running longer than the timeout are stopped and marked failed.

Timeout prevents pipelines from hanging indefinitely, saving resources and alerting you to problems.

4

Intermediate

Using Retry Option to Repeat Failed Steps

🤔Before reading on: do you think retry repeats the whole pipeline or just the step it's applied to? Commit to your answer.

Concept: Retry repeats a step a set number of times if it fails before giving up.

The retry option takes a number indicating how many times to retry. It wraps the step and reruns it if it fails. Example: pipeline { agent any stages { stage('Test') { options { retry(3) } steps { sh 'exit 1' // simulate failure } } } } This tries the test step up to 3 times before failing the stage.

Result

Failed steps are retried automatically up to the limit.

Retry helps handle flaky or temporary errors without manual intervention.

5

Intermediate

Combining Timeout and Retry for Robustness

Concept: You can use timeout and retry together to limit time and handle failures.

Options can be combined inside a stage or pipeline. For example: pipeline { agent any options { timeout(time: 5, unit: 'MINUTES') retry(2) } stages { stage('Deploy') { steps { sh './deploy.sh' } } } } This tries deploy.sh up to 2 times and stops if it runs longer than 5 minutes.

Result

Steps are retried on failure but stopped if they exceed time limits.

Combining options balances retrying errors and avoiding long hangs.

6

Advanced

Scope and Placement of Options Directive

🤔Before reading on: do you think options apply globally to the pipeline or only to the stage they are declared in? Commit to your answer.

Concept: Options can be declared at pipeline or stage level, affecting scope of their control.

When options are declared inside the pipeline block, they apply to the entire pipeline. When declared inside a stage block, they apply only to that stage. For example: pipeline { agent any options { timeout(time: 15, unit: 'MINUTES') } stages { stage('Test') { options { retry(3) } steps { sh './test.sh' } } } } Here, timeout applies to the whole pipeline, retry only to the Test stage.

Result

Options control execution scope depending on where they are declared.

Understanding scope prevents unexpected behavior and helps organize pipeline controls.

7

Expert

Handling Timeout and Retry Side Effects in Production

🤔Before reading on: do you think retry resets the timeout timer on each attempt or not? Commit to your answer.

Concept: Timeout and retry interact in subtle ways that can affect pipeline behavior in production.

When retry is used with timeout, each retry attempt resets the timeout timer. This means the total time can exceed the timeout value multiplied by retries. Also, timeout aborts the current attempt immediately, but retry may start a new attempt if retries remain. This can cause longer-than-expected pipeline runs. To manage this, experts often set conservative timeout values per attempt and monitor pipeline durations. They also handle cleanup in case of aborted steps to avoid resource leaks.

Result

Pipelines may run longer than expected if retry and timeout are combined without care.

Knowing how retry and timeout interact helps avoid hidden delays and resource issues in real pipelines.

Under the Hood

Jenkins Pipeline runs steps inside an executor process. The timeout option starts a timer when the step begins and interrupts the executor if the time limit is reached, causing the step to abort. Retry wraps the step in a loop that catches failures and reruns the step up to the retry count. Each retry attempt is a fresh execution with its own timeout timer if set. Internally, Jenkins uses Groovy CPS (Continuation Passing Style) to pause and resume pipeline execution, enabling these controls.

Why designed this way?

Timeout and retry were designed to improve pipeline reliability and resource management. Without them, pipelines could hang indefinitely or fail without recovery. The design balances simplicity and flexibility by allowing options at different scopes and combining them. Alternatives like manual error handling or external watchdogs were more complex and error-prone.

Pipeline Execution Flow
┌─────────────────────────────┐
│ Start Step                  │
│ ┌─────────────────────────┐ │
│ │ Start Timeout Timer     │ │
│ │ ┌─────────────────────┐ │ │
│ │ │ Execute Step        │ │ │
│ │ └─────────────────────┘ │ │
│ │ If Step Fails:          │ │
│ │ ┌─────────────────────┐ │ │
│ │ │ Retry Loop          │ │ │
│ │ └─────────────────────┘ │ │
│ │ If Timeout Expires:     │ │
│ │ ┌─────────────────────┐ │ │
│ │ │ Abort Step          │ │ │
│ │ └─────────────────────┘ │ │
│ └─────────────────────────┘ │
└─────────────────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does retry repeat the entire pipeline or just the step it's applied to? Commit to your answer.

Common Belief: Retry repeats the entire pipeline from the start if a step fails.

Reality: Retry only repeats the specific step or block it wraps, not the whole pipeline.

Why it matters: Misunderstanding this can cause confusion about pipeline duration and failure points.

Quick: Does timeout stop the whole pipeline immediately or just the step it's applied to? Commit to your answer.

Common Belief: Timeout always stops the entire pipeline immediately when triggered.

Reality: Timeout stops only the step or stage it is applied to; the rest of the pipeline may continue if handled.

Why it matters: Assuming full pipeline stop can lead to incorrect error handling and pipeline design.

Quick: Does retry reset the timeout timer on each attempt? Commit to your answer.

Common Belief: Retry does not affect the timeout timer; total time is limited by timeout only once.

Reality: Each retry attempt resets the timeout timer, potentially extending total run time beyond timeout × retries.

Why it matters: Ignoring this can cause pipelines to run longer than expected, wasting resources.

Quick: Can options be declared anywhere in the pipeline? Commit to your answer.

Common Belief: Options can be declared anywhere and will always apply globally.

Reality: Options have scope: declared in pipeline block applies globally; in stage block applies only to that stage.

Why it matters: Misplacing options can cause unexpected behavior and make debugging harder.

Expert Zone

1

Timeout timers reset on each retry attempt, which can cause total execution time to exceed simple calculations.

2

Retry only catches failures in the wrapped block; errors outside it are not retried, so placement matters.

3

Timeout aborts steps by interrupting the executor thread, which may leave resources in inconsistent states if not handled.

When NOT to use

Avoid using retry for steps that cause side effects like database writes without idempotency; instead, use explicit error handling or transactional controls. Timeout should not be used for very short steps where overhead outweighs benefits. For complex error recovery, consider try-catch blocks or external orchestration tools.

Production Patterns

In production, teams use timeout to prevent stuck builds and retry to handle flaky tests or network calls. They combine options with post blocks for cleanup and notifications. Some use dynamic timeout values based on step complexity. Monitoring pipeline durations helps tune these options for optimal performance.

Connections

Circuit Breaker Pattern

Similar pattern in software design to stop repeated failing operations after a threshold.

Understanding retry and timeout in Jenkins helps grasp how circuit breakers prevent cascading failures in distributed systems.

Watchdog Timer in Embedded Systems

Both timeout and watchdog timers stop processes that run too long to prevent system hangs.

Knowing how watchdog timers work in hardware clarifies why timeout is critical for pipeline stability.

Lean Manufacturing - Fail-Safe Mechanisms

Both use automatic stops and retries to prevent defects and ensure smooth production flow.

Seeing pipeline options as fail-safes connects software automation to physical process reliability.

Common Pitfalls

#1 Applying retry globally causes unexpected repeated executions.

Wrong approach:

pipeline {
agent any
options { retry(3) }
stages {
stage('Build') {
steps {
sh './build.sh'
}
}
}
}

Correct approach:

pipeline {
agent any
stages {
stage('Build') {
options { retry(3) }
steps {
sh './build.sh'
}
}
}
}

Root cause:

Retry declared at pipeline level applies to all steps, causing unintended retries.

#2 Setting timeout too short causes premature aborts.

Wrong approach:

options { timeout(time: 1, unit: 'MINUTES') } // too short for build

Correct approach:

options { timeout(time: 30, unit: 'MINUTES') } // reasonable for build

Root cause:

Misjudging step duration leads to frequent unnecessary failures.

#3 Combining retry and timeout without understanding interaction causes long runs.

Wrong approach:

options {
timeout(time: 5, unit: 'MINUTES')
retry(5)
}

Correct approach:

options {
timeout(time: 5, unit: 'MINUTES')
retry(2)
} // fewer retries to limit total time

Root cause:

Each retry resets timeout, multiplying total possible run time unexpectedly.

Key Takeaways

The Options directive in Jenkins Pipeline controls execution behavior like time limits and retries to improve reliability.

Timeout stops steps or stages that run too long, preventing stuck pipelines and wasted resources.

Retry repeats failed steps a set number of times to handle temporary errors without manual intervention.

Options can be scoped globally or per stage, affecting how and where they apply in the pipeline.

Understanding the interaction between timeout and retry is crucial to avoid unexpected pipeline durations and failures.