Combining RUN commands in Docker - Deep Dive

Overview - Combining RUN commands

What is it?

Combining RUN commands in Docker means writing multiple shell commands in a single RUN instruction inside a Dockerfile. Instead of having many separate RUN lines, you join commands using operators like && or ;. This helps create fewer layers in the final Docker image and can make the build process faster and cleaner.

Why it matters

Without combining RUN commands, each RUN creates a new image layer, making the image larger and slower to build or transfer. This wastes storage and network resources. Combining commands reduces image size and speeds up deployment, which is important for efficient software delivery and resource use.

Where it fits

Before learning this, you should understand basic Dockerfile syntax and what RUN commands do. After mastering combining RUN commands, you can learn about multi-stage builds and image optimization techniques to create even smaller and more secure Docker images.

Mental Model

Core Idea

Combining RUN commands merges multiple shell instructions into one step to reduce image layers and improve build efficiency.

Think of it like...

It's like packing several small items into one box instead of many small boxes, saving space and making transport easier.

Dockerfile Build Process
┌─────────────────────────────┐
│ Dockerfile                  │
│ ┌─────────────────────────┐ │
│ │ RUN command 1            │ │
│ └─────────────────────────┘ │
│ ┌─────────────────────────┐ │
│ │ RUN command 2            │ │
│ └─────────────────────────┘ │
│ ┌─────────────────────────┐ │
│ │ RUN command 3            │ │
│ └─────────────────────────┘ │
└───────────────┬─────────────┘
                │
                ▼
  Multiple image layers created

Combining RUN commands:

┌─────────────────────────────┐
│ Dockerfile                  │
│ ┌─────────────────────────┐ │
│ │ RUN command1 && command2 │ │
│ │ && command3             │ │
│ └─────────────────────────┘ │
└───────────────┬─────────────┘
                │
                ▼
  Single image layer created

Build-Up - 7 Steps

1

Foundation

Understanding RUN command basics

Concept: Learn what a RUN command does in a Dockerfile and how it creates image layers.

In a Dockerfile, RUN executes a command inside the image during build time. Each RUN creates a new layer, like a snapshot of the file system after the command runs. For example: RUN apt-get update RUN apt-get install -y curl creates two layers: one after updating package lists, one after installing curl.

Result

Two separate image layers are created, increasing image size.

Understanding that each RUN creates a layer helps explain why combining commands can reduce image size.

2

Foundation

How image layers affect size

Concept: Learn why each layer adds to the final image size and impacts build time.

Docker images are made of layers stacked on top of each other. Each layer stores changes from the previous one. More layers mean more storage and longer build times. For example, 5 RUN commands create 5 layers, each adding overhead.

Result

More layers increase image size and slow down build and deployment.

Knowing layers add overhead motivates combining RUN commands to reduce them.

3

Intermediate

Combining commands with && operator

🤔Before reading on: do you think using '&&' between commands runs all commands even if one fails, or stops at the first failure? Commit to your answer.

Concept: Use '&&' to chain commands so the next runs only if the previous succeeds.

You can write multiple commands in one RUN line using '&&'. For example: RUN apt-get update && apt-get install -y curl This runs 'apt-get update' first. If it succeeds, then 'apt-get install -y curl' runs. If the first fails, the build stops.

Result

One RUN layer is created, and the build fails early if a command fails.

Using '&&' ensures commands run safely in sequence, preventing partial or broken builds.

4

Intermediate

Using line continuation for readability

🤔Before reading on: do you think splitting a RUN command over multiple lines affects the number of layers created? Commit to your answer.

Concept: Use backslash '\' to split long RUN commands over multiple lines without creating extra layers.

Long RUN commands can be hard to read. Use '\' to continue lines: RUN apt-get update && \ apt-get install -y curl && \ rm -rf /var/lib/apt/lists/* This is still one RUN command and creates one layer.

Result

Improved readability with no extra image layers.

Knowing line continuation keeps Dockerfiles clean without affecting build layers.

5

Intermediate

Combining commands with ; operator

🤔Before reading on: does using ';' between commands stop execution on failure or continue regardless? Commit to your answer.

Concept: ';' runs commands sequentially regardless of success or failure.

You can separate commands with ';' like: RUN command1; command2; command3 All commands run one after another even if one fails. This can cause unexpected results if a command fails silently.

Result

One RUN layer is created but errors may be ignored.

Understanding ';' behavior helps avoid silent failures in builds.

6

Advanced

Reducing image size with cleanup commands

🤔Before reading on: do you think cleanup commands should be in separate RUN steps or combined with install commands? Commit to your answer.

Concept: Combine install and cleanup commands in one RUN to avoid leftover files in intermediate layers.

If you install packages and then clean cache in separate RUN commands, cache remains in earlier layers. Instead, combine: RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/* This keeps the image smaller by removing temporary files in the same layer.

Result

Smaller final image size with no leftover cache files.

Knowing how layers store files prevents common mistakes that bloat images.

7

Expert

Subtle effects of combining RUN on caching

🤔Before reading on: does combining many commands in one RUN always speed up builds? Commit to your answer.

Concept: Combining RUN commands affects Docker build cache behavior, which can speed up or slow down builds depending on changes.

Docker caches each layer. If you combine many commands, any change in one command invalidates the whole RUN cache, causing all commands to rerun. Splitting commands can allow partial caching. For example: RUN apt-get update && apt-get install -y curl If you change install command, update runs again too. Splitting can save time but increases layers. Balancing combining commands and caching is key for efficient builds.

Result

Build speed depends on how RUN commands are combined and changed.

Understanding cache invalidation helps optimize build speed and image size trade-offs.

Under the Hood

Each RUN command creates a new image layer by executing the shell command in a temporary container. The resulting file system changes are saved as a layer. Combining commands in one RUN means all commands run in one shell session, producing one layer with all changes combined. Docker caches layers by hashing the command and previous layers, so changes in combined commands invalidate the entire layer cache.

Why designed this way?

Docker layers allow efficient reuse and sharing of image parts. Creating a layer per RUN command simplifies caching and rollback. However, too many layers increase image size and overhead. Combining RUN commands balances layer count and build efficiency. The design trades off simplicity of caching with image size optimization.

Docker Build Layers

Dockerfile:
RUN cmd1
RUN cmd2
RUN cmd3

Build process:
┌───────────────┐
│ Base Image    │
└──────┬────────┘
       │
┌──────▼───────┐
│ Layer 1: cmd1│
└──────┬───────┘
       │
┌──────▼───────┐
│ Layer 2: cmd2│
└──────┬───────┘
       │
┌──────▼───────┐
│ Layer 3: cmd3│
└──────────────┘

Combined RUN:
RUN cmd1 && cmd2 && cmd3

Build process:
┌───────────────┐
│ Base Image    │
└──────┬────────┘
       │
┌──────▼─────────────┐
│ Layer 1: cmd1+cmd2+cmd3 │
└─────────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does combining RUN commands always make the build faster? Commit to yes or no before reading on.

Common Belief: Combining RUN commands always speeds up Docker builds.

Reality: Combining commands can slow builds if any part changes, because the entire combined layer cache is invalidated and must rerun.

Why it matters: Assuming combining always speeds builds can lead to slower development cycles and wasted time.

Quick: Does using ';' between commands stop execution on failure? Commit to yes or no before reading on.

Common Belief: Using ';' between commands in RUN stops execution if a command fails.

Reality: ';' runs all commands regardless of failure, which can hide errors and cause broken images.

Why it matters: Ignoring failures can cause images to build successfully but fail at runtime, wasting debugging time.

Quick: Does splitting RUN commands over multiple lines create multiple layers? Commit to yes or no before reading on.

Common Belief: Splitting RUN commands over multiple lines creates multiple image layers.

Reality: Using '\' to split lines is just for readability; it still creates one layer.

Why it matters: Misunderstanding this leads to unnecessarily long single-line commands or bloated images.

Quick: Can cleanup commands be run in separate RUN steps without affecting image size? Commit to yes or no before reading on.

Common Belief: Running cleanup commands in separate RUN steps reduces image size just as well as combining them.

Reality: Separate RUN steps keep temporary files in earlier layers, increasing image size.

Why it matters: Not combining cleanup with install commands leads to larger images and slower deployments.

Expert Zone

1

Combining RUN commands affects Docker layer caching granularity, impacting build speed and cache reuse.

2

Order of commands in combined RUN matters; placing frequently changing commands last preserves cache for earlier steps.

3

Using shell features like set -eux can improve error handling and debugging in combined RUN commands.

When NOT to use

Avoid combining RUN commands when individual commands change frequently and you want to leverage Docker's layer caching for faster incremental builds. Instead, split commands into separate RUN steps to maximize cache reuse.

Production Patterns

In production Dockerfiles, combining RUN commands is used to install packages and clean caches in one step to minimize image size. Multi-stage builds often combine RUN commands for build dependencies, then copy only artifacts to final image. Scripts or Makefiles may be called inside RUN to keep Dockerfile clean.

Connections

Layered File Systems

Builds-on

Understanding how Docker layers build on each other clarifies why combining RUN commands reduces image size.

Shell Scripting

Same pattern

Combining commands with && and ; in Docker RUN mirrors shell scripting practices, so shell knowledge improves Dockerfile writing.

Supply Chain Optimization

Analogy

Just like combining shipments reduces transport costs, combining RUN commands reduces image size and speeds delivery.

Common Pitfalls

#1 Ignoring command failure in combined RUN commands.

Wrong approach:

RUN apt-get update; apt-get install -y curl; echo 'Done'

Correct approach:

RUN apt-get update && apt-get install -y curl && echo 'Done'

Root cause:

Using ';' runs all commands regardless of failure, hiding errors.

#2 Running cleanup commands in separate RUN steps.

Wrong approach:

RUN apt-get update && apt-get install -y curl
RUN rm -rf /var/lib/apt/lists/*

Correct approach:

RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/*

Root cause:

Separate RUN steps keep temporary files in earlier layers, increasing image size.

#3 Writing very long RUN commands without line breaks.

Wrong approach:

RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/* && echo 'Done' && other commands

Correct approach:

RUN apt-get update && \
apt-get install -y curl && \
rm -rf /var/lib/apt/lists/* && \
echo 'Done' && \
other commands

Root cause:

Not using line continuation reduces readability and maintainability.

Key Takeaways

Each RUN command in a Dockerfile creates a new image layer, increasing image size and build time.

Combining multiple commands into a single RUN instruction reduces the number of layers and results in smaller images.

Using '&&' between commands ensures that subsequent commands run only if previous ones succeed, preventing broken builds.

Splitting long RUN commands over multiple lines with '\' improves readability without adding layers.

Balancing command combination and Docker cache behavior is key to optimizing build speed and image size.