Combining RUN commands in Docker - Deep Dive

Overview - Combining RUN commands

What is it?

Combining RUN commands in Docker means writing multiple shell commands in a single RUN instruction inside a Dockerfile. Instead of having many separate RUN lines, you join commands using operators like && or ;. This helps create fewer layers in the final Docker image and can make the build process faster and cleaner.

Why it matters

Without combining RUN commands, each RUN creates a new image layer, making the image larger and slower to build or transfer. This wastes storage and network resources. Combining commands reduces image size and speeds up deployment, which is important for efficient software delivery and resource use.

Where it fits

Before learning this, you should understand basic Dockerfile syntax and what RUN commands do. After mastering combining RUN commands, you can learn about multi-stage builds and image optimization techniques to create even smaller and more secure Docker images.

Mental Model

Core Idea

Combining RUN commands merges multiple shell instructions into one step to reduce image layers and improve build efficiency.

Think of it like...

It's like packing several small items into one box instead of many small boxes, saving space and making transport easier.

Dockerfile Build Process
┌─────────────────────────────┐
│ Dockerfile                  │
│ ┌─────────────────────────┐ │
│ │ RUN command 1            │ │
│ └─────────────────────────┘ │
│ ┌─────────────────────────┐ │
│ │ RUN command 2            │ │
│ └─────────────────────────┘ │
│ ┌─────────────────────────┐ │
│ │ RUN command 3            │ │
│ └─────────────────────────┘ │
└───────────────┬─────────────┘
                │
                ▼
  Multiple image layers created

Combining RUN commands:

┌─────────────────────────────┐
│ Dockerfile                  │
│ ┌─────────────────────────┐ │
│ │ RUN command1 && command2 │ │
│ │ && command3             │ │
│ └─────────────────────────┘ │
└───────────────┬─────────────┘
                │
                ▼
  Single image layer created

Build-Up - 7 Steps

1

Foundation

Understanding RUN command basics

Concept: Learn what a RUN command does in a Dockerfile and how it creates image layers.

In a Dockerfile, RUN executes a command inside the image during build time. Each RUN creates a new layer, like a snapshot of the file system after the command runs. For example: RUN apt-get update RUN apt-get install -y curl creates two layers: one after updating package lists, one after installing curl.

Result

Two separate image layers are created, increasing image size.

Understanding that each RUN creates a layer helps explain why combining commands can reduce image size.

2

Foundation

How image layers affect size

Concept: Learn why each layer adds to the final image size and impacts build time.

Docker images are made of layers stacked on top of each other. Each layer stores changes from the previous one. More layers mean more storage and longer build times. For example, 5 RUN commands create 5 layers, each adding overhead.

Result

More layers increase image size and slow down build and deployment.

Knowing layers add overhead motivates combining RUN commands to reduce them.

3

Intermediate

Combining commands with && operator

🤔Before reading on: do you think using '&&' between commands runs all commands even if one fails, or stops at the first failure? Commit to your answer.

Concept: Use '&&' to chain commands so the next runs only if the previous succeeds.

You can write multiple commands in one RUN line using '&&'. For example: RUN apt-get update && apt-get install -y curl This runs 'apt-get update' first. If it succeeds, then 'apt-get install -y curl' runs. If the first fails, the build stops.

Result

One RUN layer is created, and the build fails early if a command fails.

Using '&&' ensures commands run safely in sequence, preventing partial or broken builds.

4

Intermediate

Using line continuation for readability

🤔Before reading on: do you think splitting a RUN command over multiple lines affects the number of layers created? Commit to your answer.

Concept: Use backslash '\' to split long RUN commands over multiple lines without creating extra layers.

Long RUN commands can be hard to read. Use '\' to continue lines: RUN apt-get update && \ apt-get install -y curl && \ rm -rf /var/lib/apt/lists/* This is still one RUN command and creates one layer.

Result

Improved readability with no extra image layers.

Knowing line continuation keeps Dockerfiles clean without affecting build layers.

5

Intermediate

Combining commands with ; operator

🤔Before reading on: does using ';' between commands stop execution on failure or continue regardless? Commit to your answer.

Concept: ';' runs commands sequentially regardless of success or failure.

You can separate commands with ';' like: RUN command1; command2; command3 All commands run one after another even if one fails. This can cause unexpected results if a command fails silently.

Result

One RUN layer is created but errors may be ignored.

Understanding ';' behavior helps avoid silent failures in builds.

6

Advanced

Reducing image size with cleanup commands

🤔Before reading on: do you think cleanup commands should be in separate RUN steps or combined with install commands? Commit to your answer.

Concept: Combine install and cleanup commands in one RUN to avoid leftover files in intermediate layers.

If you install packages and then clean cache in separate RUN commands, cache remains in earlier layers. Instead, combine: RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/* This keeps the image smaller by removing temporary files in the same layer.

Result

Smaller final image size with no leftover cache files.

Knowing how layers store files prevents common mistakes that bloat images.

7

Expert

Subtle effects of combining RUN on caching

🤔Before reading on: does combining many commands in one RUN always speed up builds? Commit to your answer.

Concept: Combining RUN commands affects Docker build cache behavior, which can speed up or slow down builds depending on changes.

Docker caches each layer. If you combine many commands, any change in one command invalidates the whole RUN cache, causing all commands to rerun. Splitting commands can allow partial caching. For example: RUN apt-get update && apt-get install -y curl If you change install command, update runs again too. Splitting can save time but increases layers. Balancing combining commands and caching is key for efficient builds.

Result

Build speed depends on how RUN commands are combined and changed.

Understanding cache invalidation helps optimize build speed and image size trade-offs.

Under the Hood

Each RUN command creates a new image layer by executing the shell command in a temporary container. The resulting file system changes are saved as a layer. Combining commands in one RUN means all commands run in one shell session, producing one layer with all changes combined. Docker caches layers by hashing the command and previous layers, so changes in combined commands invalidate the entire layer cache.

Why designed this way?

Docker layers allow efficient reuse and sharing of image parts. Creating a layer per RUN command simplifies caching and rollback. However, too many layers increase image size and overhead. Combining RUN commands balances layer count and build efficiency. The design trades off simplicity of caching with image size optimization.

Docker Build Layers

Dockerfile:
RUN cmd1
RUN cmd2
RUN cmd3

Build process:
┌───────────────┐
│ Base Image    │
└──────┬────────┘
       │
┌──────▼───────┐
│ Layer 1: cmd1│
└──────┬───────┘
       │
┌──────▼───────┐
│ Layer 2: cmd2│
└──────┬───────┘
       │
┌──────▼───────┐
│ Layer 3: cmd3│
└──────────────┘

Combined RUN:
RUN cmd1 && cmd2 && cmd3

Build process:
┌───────────────┐
│ Base Image    │
└──────┬────────┘
       │
┌──────▼─────────────┐
│ Layer 1: cmd1+cmd2+cmd3 │
└─────────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does combining RUN commands always make the build faster? Commit to yes or no before reading on.

Common Belief: Combining RUN commands always speeds up Docker builds.

Reality: Combining commands can slow builds if any part changes, because the entire combined layer cache is invalidated and must rerun.

Why it matters: Assuming combining always speeds builds can lead to slower development cycles and wasted time.

Quick: Does using ';' between commands stop execution on failure? Commit to yes or no before reading on.

Common Belief: Using ';' between commands in RUN stops execution if a command fails.

Reality: ';' runs all commands regardless of failure, which can hide errors and cause broken images.

Why it matters: Ignoring failures can cause images to build successfully but fail at runtime, wasting debugging time.

Quick: Does splitting RUN commands over multiple lines create multiple layers? Commit to yes or no before reading on.

Common Belief: Splitting RUN commands over multiple lines creates multiple image layers.

Reality: Using '\' to split lines is just for readability; it still creates one layer.

Why it matters: Misunderstanding this leads to unnecessarily long single-line commands or bloated images.

Quick: Can cleanup commands be run in separate RUN steps without affecting image size? Commit to yes or no before reading on.

Common Belief: Running cleanup commands in separate RUN steps reduces image size just as well as combining them.

Reality: Separate RUN steps keep temporary files in earlier layers, increasing image size.

Why it matters: Not combining cleanup with install commands leads to larger images and slower deployments.

Expert Zone

1

Combining RUN commands affects Docker layer caching granularity, impacting build speed and cache reuse.

2

Order of commands in combined RUN matters; placing frequently changing commands last preserves cache for earlier steps.

3

Using shell features like set -eux can improve error handling and debugging in combined RUN commands.

When NOT to use

Avoid combining RUN commands when individual commands change frequently and you want to leverage Docker's layer caching for faster incremental builds. Instead, split commands into separate RUN steps to maximize cache reuse.

Production Patterns

In production Dockerfiles, combining RUN commands is used to install packages and clean caches in one step to minimize image size. Multi-stage builds often combine RUN commands for build dependencies, then copy only artifacts to final image. Scripts or Makefiles may be called inside RUN to keep Dockerfile clean.

Connections

Layered File Systems

Builds-on

Understanding how Docker layers build on each other clarifies why combining RUN commands reduces image size.

Shell Scripting

Same pattern

Combining commands with && and ; in Docker RUN mirrors shell scripting practices, so shell knowledge improves Dockerfile writing.

Supply Chain Optimization

Analogy

Just like combining shipments reduces transport costs, combining RUN commands reduces image size and speeds delivery.

Common Pitfalls

#1 Ignoring command failure in combined RUN commands.

Wrong approach:

RUN apt-get update; apt-get install -y curl; echo 'Done'

Correct approach:

RUN apt-get update && apt-get install -y curl && echo 'Done'

Root cause:

Using ';' runs all commands regardless of failure, hiding errors.

#2 Running cleanup commands in separate RUN steps.

Wrong approach:

RUN apt-get update && apt-get install -y curl
RUN rm -rf /var/lib/apt/lists/*

Correct approach:

RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/*

Root cause:

Separate RUN steps keep temporary files in earlier layers, increasing image size.

#3 Writing very long RUN commands without line breaks.

Wrong approach:

RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/* && echo 'Done' && other commands

Correct approach:

RUN apt-get update && \
apt-get install -y curl && \
rm -rf /var/lib/apt/lists/* && \
echo 'Done' && \
other commands

Root cause:

Not using line continuation reduces readability and maintainability.

Key Takeaways

Each RUN command in a Dockerfile creates a new image layer, increasing image size and build time.

Combining multiple commands into a single RUN instruction reduces the number of layers and results in smaller images.

Using '&&' between commands ensures that subsequent commands run only if previous ones succeed, preventing broken builds.

Splitting long RUN commands over multiple lines with '\' improves readability without adding layers.

Balancing command combination and Docker cache behavior is key to optimizing build speed and image size.

Practice

1. Why is it recommended to combine multiple commands in a single RUN instruction in a Dockerfile?

easy

A. To make the Dockerfile longer and more complex

B. To run commands in parallel for faster builds

C. To reduce the number of image layers and keep the image size smaller

D. To avoid using the shell in Dockerfile commands

Solution

1. Step 1: Understand Docker image layers

   Each RUN instruction creates a new image layer, increasing image size.

2. Step 2: Combine commands to reduce layers

   Combining commands with && in one RUN reduces layers and image size.

3. Final Answer:

   To reduce the number of image layers and keep the image size smaller -> Option C

4. Quick Check:

   Combining RUN commands = fewer layers [OK]

Hint: Combine commands with && to reduce image layers [OK]

Common Mistakes:

• Thinking combining commands makes builds faster
• Believing each command runs in parallel
• Ignoring image size impact of multiple RUNs

2. Which of the following is the correct syntax to combine two commands in a single RUN instruction in a Dockerfile?

easy

A. RUN apt-get update && apt-get install -y curl

B. RUN apt-get update; apt-get install -y curl

C. RUN apt-get update | apt-get install -y curl

D. RUN apt-get update &&& apt-get install -y curl

Solution

1. Step 1: Identify correct command chaining operator

   The && operator chains commands so the second runs only if the first succeeds.

2. Step 2: Check each option's syntax

   RUN apt-get update && apt-get install -y curl uses && correctly; others use invalid or unsafe separators.

3. Final Answer:

   RUN apt-get update && apt-get install -y curl -> Option A

4. Quick Check:

   Use && to chain commands safely [OK]

Hint: Use && to chain commands safely in RUN [OK]

Common Mistakes:

• Using semicolon without error checking
• Using invalid operators like | or &&&
• Not chaining commands properly causing build errors

3. What will be the result of this Dockerfile snippet?

RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/*

medium

A. Creates three separate layers for each command

B. Only updates package lists and installs curl, but does not clean files

C. Fails because rm command is invalid in RUN

D. Updates package lists, installs curl, and cleans temporary files in one layer

Solution

1. Step 1: Analyze the chained RUN commands

   The commands update packages, install curl, then remove apt cache files in one RUN.

2. Step 2: Understand layer creation

   All commands combined in one RUN create a single image layer with all changes.

3. Final Answer:

   Updates package lists, installs curl, and cleans temporary files in one layer -> Option D

4. Quick Check:

   Combined RUN commands = single layer with all steps [OK]

Hint: One RUN with && creates one layer with all commands [OK]

Common Mistakes:

• Thinking each command creates a separate layer
• Assuming rm command is invalid in RUN
• Ignoring cleaning temporary files effect

4. You wrote this Dockerfile line:

RUN apt-get update & apt-get install -y git

What is the problem with this command?

medium

A. The & operator is correct and recommended for RUN commands

B. The single & runs commands in background, causing build errors

C. The commands will run sequentially without errors

D. The command will install git twice

Solution

1. Step 1: Understand the & operator in shell

   Single & runs the first command in background, not waiting for completion.

2. Step 2: Impact on Docker build

   This causes the second command to run immediately, possibly before update finishes, leading to errors.

3. Final Answer:

   The single & runs commands in background, causing build errors -> Option B

4. Quick Check:

   Use && not & to chain RUN commands [OK]

Hint: Use &&, not &, to chain commands in RUN [OK]

Common Mistakes:

• Using & instead of && for chaining
• Assuming commands run sequentially with &
• Ignoring build errors caused by background commands

5. You want to install curl and git in one RUN command and clean up apt cache to keep the image small. Which Dockerfile line correctly does this?

hard

A. RUN apt-get update && apt-get install -y curl git && rm -rf /var/lib/apt/lists/*

B. RUN apt-get update; apt-get install -y curl git; rm -rf /var/lib/apt/lists/*

C. RUN apt-get update && apt-get install -y curl && apt-get install -y git && rm -rf /var/lib/apt/lists/*

D. RUN apt-get update & apt-get install -y curl git && rm -rf /var/lib/apt/lists/*

Solution

1. Step 1: Combine update and install commands safely

   Use && to chain update and install commands to ensure sequential success.

2. Step 2: Clean apt cache in the same RUN

   Remove temporary files with rm -rf /var/lib/apt/lists/* in the same RUN to keep image small.

3. Step 3: Check options for correctness

   RUN apt-get update && apt-get install -y curl git && rm -rf /var/lib/apt/lists/* chains all commands correctly in one RUN; others use semicolons or background & causing issues or inefficiencies.

4. Final Answer:

   RUN apt-get update && apt-get install -y curl git && rm -rf /var/lib/apt/lists/* -> Option A

5. Quick Check:

   Chain with && and clean in one RUN [OK]

Hint: Chain update, install, and cleanup with && in one RUN [OK]

Common Mistakes:

• Using semicolons instead of && without error checking
• Running install commands separately (inefficient)
• Using & causing background execution errors