Restart policies in Docker - Deep Dive

Overview - Restart policies

What is it?

Restart policies in Docker are rules that tell containers when and how to restart automatically. They help keep containers running smoothly by restarting them if they stop unexpectedly or after a system reboot. This ensures your applications stay available without manual intervention. Restart policies are simple settings you add when you create or run a container.

Why it matters

Without restart policies, if a container crashes or the host machine restarts, your application would stay down until someone notices and restarts it manually. This can cause downtime and lost users or data. Restart policies automate recovery, making your services more reliable and reducing the need for constant monitoring.

Where it fits

Before learning restart policies, you should understand basic Docker container creation and management. After mastering restart policies, you can explore Docker Compose for managing multi-container setups and Kubernetes for advanced container orchestration.

Mental Model

Core Idea

Restart policies are simple rules that tell Docker when to automatically restart containers to keep applications running reliably.

Think of it like...

It's like a smoke alarm in your home that automatically calls for help when it detects smoke, so you don't have to watch the stove constantly.

┌─────────────────────────────┐
│       Docker Container       │
├─────────────┬───────────────┤
│   Runs App  │  Stops/Crashes│
├─────────────┴───────────────┤
│ Restart Policy Checks Status │
├─────────────┬───────────────┤
│ Always      │ Restart Always│
│ On-failure  │ Restart if Fail│
│ Unless-stopped│ Restart unless stopped│
│ No         │ No Restart    │
└─────────────────────────────┘

Build-Up - 7 Steps

1

Foundation

What are Docker restart policies

Concept: Introduce the basic idea of restart policies and their purpose.

Docker restart policies are settings you add when starting a container to tell Docker if and when it should restart that container automatically. They help keep your app running without manual restarts if it crashes or the system reboots.

Result

You understand that restart policies are simple rules that improve container reliability by automating restarts.

Knowing that restart policies exist helps you avoid downtime caused by unexpected container stops.

2

Foundation

Types of restart policies available

Concept: Learn the four main restart policies Docker supports and what each means.

Docker supports these restart policies: - no: Do not restart automatically. - always: Restart always, even if stopped manually. - unless-stopped: Restart unless the container was stopped by the user. - on-failure: Restart only if the container exits with an error. You set these with the --restart flag when running a container.

Result

You can name and explain the four restart policies Docker offers.

Understanding the differences lets you choose the right policy for your app's needs.

3

Intermediate

Using restart policies with docker run

🤔Before reading on: do you think the 'always' policy restarts containers stopped manually? Commit to your answer.

Concept: Learn how to apply restart policies using the docker run command.

To use a restart policy, add --restart= to your docker run command. For example: docker run --restart=always nginx This runs an nginx container that Docker will always restart if it stops or the system reboots.

Result

You can start containers with restart policies that automate restarts.

Knowing the exact command syntax lets you quickly add reliability to any container.

4

Intermediate

How restart policies behave on system reboot

🤔Before reading on: do you think 'on-failure' restart policy restarts containers after a system reboot? Commit to your answer.

Concept: Understand how restart policies affect containers when the host machine restarts.

When the host restarts, Docker checks containers with restart policies: - always and unless-stopped containers restart automatically. - on-failure containers restart only if they stopped with an error before reboot. - no policy containers do not restart. This helps keep critical services running after reboots.

Result

You know which containers come back after a reboot based on their restart policy.

Understanding reboot behavior helps you plan for system maintenance without downtime.

5

Intermediate

Combining restart policies with health checks

🤔Before reading on: do you think Docker restart policies restart containers based on health check failures? Commit to your answer.

Concept: Learn how restart policies interact with Docker health checks to improve reliability.

Docker health checks run commands inside containers to check if the app is healthy. Restart policies alone restart containers only if they stop. If a container is unhealthy but still running, restart policies do NOT restart it automatically. You can combine health checks with external tools to restart unhealthy containers.

Result

You understand that restart policies do not cover all failure cases alone.

Knowing this prevents false assumptions about container reliability and guides better monitoring setups.

6

Advanced

Limitations and edge cases of restart policies

🤔Before reading on: do you think 'always' restart policy can cause infinite restart loops? Commit to your answer.

Concept: Explore situations where restart policies might cause problems or behave unexpectedly.

If a container crashes immediately after starting, 'always' or 'on-failure' policies can cause rapid restart loops, consuming resources. Docker tries to limit this by backing off restarts after repeated failures. Also, manual stops behave differently with 'always' vs 'unless-stopped'. Understanding these helps avoid resource exhaustion and unexpected behavior.

Result

You can predict and prevent restart policy pitfalls in production.

Knowing these edge cases helps you design safer container restart strategies.

7

Expert

How Docker implements restart policies internally

🤔Before reading on: do you think Docker restart policies are managed by the container runtime or the Docker daemon? Commit to your answer.

Concept: Understand the internal mechanism Docker uses to monitor and restart containers based on policies.

Docker daemon tracks container states and exit codes. It uses an event loop to watch containers and applies restart policies. Restart decisions are made by the daemon, not the container runtime. Docker also uses backoff timers to avoid rapid restart loops. This design centralizes control and simplifies management.

Result

You grasp the internal process behind automatic container restarts.

Understanding the daemon's role clarifies why restart policies behave consistently across containers.

Under the Hood

The Docker daemon continuously monitors container states and exit codes. When a container stops, the daemon checks the assigned restart policy and decides whether to restart it. It uses timers to prevent rapid restart loops by delaying restarts after repeated failures. The daemon handles restarts centrally, independent of the container runtime, ensuring consistent behavior.

Why designed this way?

Centralizing restart logic in the Docker daemon simplifies container lifecycle management and avoids complexity inside containers. This design allows Docker to uniformly apply policies across all containers and handle edge cases like rapid failures with backoff timers. Alternatives like embedding restart logic inside containers would be less reliable and harder to manage.

┌───────────────┐       ┌───────────────┐
│ Container    │       │ Docker Daemon │
│ Runs App     │──────▶│ Monitors State│
│ Exits/Stops │       │ Checks Policy │
└───────────────┘       └──────┬────────┘
                                │
                                ▼
                      ┌───────────────────┐
                      │ Restart Decision  │
                      │ (based on policy) │
                      └─────────┬─────────┘
                                │
                    ┌───────────┴───────────┐
                    │ Restart Container     │
                    │ or Do Nothing         │
                    └───────────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does the 'always' restart policy restart containers stopped manually? Commit to yes or no.

Common Belief: The 'always' restart policy restarts containers even if you stopped them manually.

Reality: The 'always' policy restarts containers regardless of how they stopped, including manual stops.

Why it matters: This can cause confusion and frustration if you try to stop a container but it keeps restarting unexpectedly.

Quick: Will 'on-failure' restart containers after a system reboot? Commit to yes or no.

Common Belief: 'On-failure' restart policy restarts containers after the host machine reboots.

Reality: 'On-failure' only restarts containers if they exited with an error before reboot; it does not restart containers that were stopped cleanly.

Why it matters: Assuming 'on-failure' always restarts after reboot can lead to unexpected downtime.

Quick: Do restart policies restart containers when health checks fail? Commit to yes or no.

Common Belief: Docker restart policies restart containers automatically if their health checks fail.

Reality: Restart policies only restart containers when they stop; health check failures alone do not trigger restarts.

Why it matters: Relying solely on restart policies for health issues can leave unhealthy containers running.

Quick: Can restart policies cause infinite restart loops? Commit to yes or no.

Common Belief: Restart policies always safely restart containers without causing resource issues.

Reality: Rapid crash and restart cycles can cause infinite loops, but Docker uses backoff timers to reduce this risk.

Why it matters: Ignoring this can cause high CPU and memory usage, affecting system stability.

Expert Zone

1

The 'unless-stopped' policy respects manual stops but still restarts containers after reboots, unlike 'always'.

2

Docker's backoff timer for restart attempts grows exponentially to prevent resource exhaustion during crash loops.

3

Restart policies do not affect containers managed by orchestration tools like Kubernetes, which handle restarts differently.

When NOT to use

Restart policies are not suitable for complex multi-container applications requiring coordinated restarts; use Docker Compose or Kubernetes instead. Also, do not rely on restart policies alone for health monitoring; combine with health checks and external monitoring tools.

Production Patterns

In production, 'unless-stopped' is commonly used for services that should run continuously but allow manual stops. 'On-failure' is used for batch jobs where restarts only happen on errors. Operators combine restart policies with health checks and alerting systems to maintain high availability.

Connections

Systemd service restart options

Similar pattern

Both Docker restart policies and systemd service restart options automate recovery of processes, showing a common approach to reliability in software management.

Fault tolerance in distributed systems

Builds-on

Restart policies are a basic form of fault tolerance, helping systems recover from failures automatically, which is a key principle in distributed system design.

Biological homeostasis

Analogous process

Just like living organisms maintain stable internal conditions automatically, restart policies maintain stable application states by restarting containers when needed.

Common Pitfalls

#1 Expecting containers to restart on health check failures alone.

Wrong approach:

docker run --restart=always myapp
# Assuming this restarts container if health check fails

Correct approach:

Use health checks combined with external monitoring or orchestrators to restart unhealthy containers, as restart policies only restart on container exit.

Root cause:

Misunderstanding that restart policies monitor container exit status, not health check results.

#2 Using 'always' restart policy without considering crash loops.

Wrong approach:

docker run --restart=always buggyapp
# Container crashes immediately and restarts endlessly

Correct approach:

Use 'on-failure' with max retry limits or fix the underlying crash to avoid resource exhaustion.

Root cause:

Not accounting for rapid failure and restart cycles causing infinite loops.

#3 Stopping a container with 'always' restart policy and expecting it to stay stopped.

Wrong approach:

docker stop mycontainer
# Container restarts automatically despite stop command

Correct approach:

Use 'unless-stopped' policy if you want manual stops to persist without automatic restarts.

Root cause:

Confusing behavior differences between 'always' and 'unless-stopped' policies.

Key Takeaways

Docker restart policies automate container restarts to improve application reliability and reduce downtime.

There are four main restart policies: no, always, unless-stopped, and on-failure, each with distinct behaviors.

Restart policies only restart containers when they stop; they do not react to health check failures alone.

Understanding how restart policies behave during system reboots and manual stops is crucial to avoid surprises.

Combining restart policies with health checks and monitoring tools creates robust, production-ready container setups.