RUN instruction for executing commands in Docker - Deep Dive

Overview - RUN instruction for executing commands

What is it?

The RUN instruction in Dockerfiles is used to execute commands inside a new layer of the Docker image during the build process. It allows you to install software, update packages, or configure the environment before the image is finalized. Each RUN command creates a new image layer that becomes part of the final container image.

Why it matters

Without the RUN instruction, you would have to manually configure your container environment every time you start a container, which is slow and error-prone. RUN automates setup steps, making images reusable and consistent. This saves time, reduces mistakes, and ensures your applications run the same everywhere.

Where it fits

Before learning RUN, you should understand basic Docker concepts like images, containers, and Dockerfiles. After mastering RUN, you can learn about other Dockerfile instructions like CMD, ENTRYPOINT, and how to optimize image layers for smaller, faster builds.

Mental Model

Core Idea

RUN executes commands during image build to prepare the container environment layer by layer.

Think of it like...

RUN is like a chef following a recipe step-by-step to prepare ingredients before cooking the final dish. Each step adds something new to the dish, building up the final flavor.

Dockerfile Build Process:

  ┌─────────────┐
  │ Base Image  │
  └─────┬───────┘
        │
  ┌─────▼───────┐
  │ RUN Command │  <-- Executes command, creates new layer
  └─────┬───────┘
        │
  ┌─────▼───────┐
  │ New Image   │  <-- Includes changes from RUN
  └─────────────┘

Build-Up - 7 Steps

1

Foundation

What is the RUN instruction

Concept: RUN lets you run commands inside the image while building it.

In a Dockerfile, RUN is followed by a command you want to execute. For example, RUN apt-get update will update package lists inside the image. This happens during build, not when the container runs.

Result

The image includes the results of the command, like updated packages or installed software.

Understanding RUN is key because it shapes the environment your container will have when it runs.

2

Foundation

How RUN creates image layers

Concept: Each RUN command creates a new layer in the Docker image.

Docker images are made of layers stacked on top of each other. When you use RUN, Docker executes the command and saves the changes as a new layer. Layers help Docker reuse parts of images to save space and speed up builds.

Result

The image grows with each RUN, but layers can be reused if unchanged.

Knowing that RUN creates layers helps you write efficient Dockerfiles by minimizing unnecessary layers.

3

Intermediate

RUN syntax: shell vs exec form

🤔Before reading on: do you think RUN commands always run in a shell or can they run directly? Commit to your answer.

Concept: RUN supports two syntax styles: shell form and exec form, which affect how commands run.

Shell form looks like: RUN apt-get update Exec form looks like: RUN ["apt-get", "update"] Shell form runs commands inside a shell like /bin/sh -c, allowing shell features. Exec form runs the command directly without a shell.

Result

Shell form allows shell features like variable expansion; exec form is more precise and avoids shell parsing.

Choosing the right RUN syntax affects command behavior and security, especially when handling complex commands or arguments.

4

Intermediate

Combining multiple commands in RUN

🤔Before reading on: do you think multiple commands in RUN create multiple layers or one layer? Commit to your answer.

Concept: You can combine several commands in one RUN to reduce image layers.

Using shell operators like && lets you chain commands: RUN apt-get update && apt-get install -y curl This runs both commands in one RUN, creating a single layer instead of two.

Result

The image has fewer layers, which can make it smaller and faster to build.

Combining commands in RUN helps optimize image size and build speed by reducing layers.

5

Intermediate

Using RUN to install software packages

Concept: RUN is commonly used to install software needed by your app inside the image.

For example, RUN apt-get update && apt-get install -y python3 installs Python3 in the image. This prepares the environment so your app can run without missing dependencies.

Result

The image contains the installed software, ready for use in containers.

Knowing how to install software with RUN is essential for creating functional container images.

6

Advanced

Cleaning up after RUN to reduce image size

🤔Before reading on: do you think files created during RUN persist in the final image even if deleted later? Commit to your answer.

Concept: Files created and deleted in separate RUN commands still add to image size; cleaning must happen in the same RUN.

If you run: RUN apt-get update RUN apt-get clean The update files remain in the image because each RUN is a separate layer. Instead, combine cleanup: RUN apt-get update && apt-get install -y curl && apt-get clean && rm -rf /var/lib/apt/lists/* This keeps the image smaller.

Result

The final image is smaller and more efficient.

Understanding how layers work prevents accidentally bloating images with temporary files.

7

Expert

RUN caching and build performance surprises

🤔Before reading on: do you think changing one RUN command invalidates all following layers or only that layer? Commit to your answer.

Concept: Docker caches RUN layers to speed up builds, but cache invalidation can cause unexpected rebuilds.

Docker reuses cached layers if the RUN command and previous layers are unchanged. Changing a RUN command invalidates its layer and all layers after it, causing rebuilds. This means ordering RUN commands carefully can optimize build times. Also, commands that depend on external state (like apt-get update) can cause cache misses if not handled properly.

Result

Efficient Dockerfiles build faster by leveraging cache; inefficient ones rebuild unnecessarily.

Mastering RUN caching behavior is crucial for fast, reliable Docker builds in production.

Under the Hood

When Docker processes a RUN instruction, it creates a temporary container from the current image state, executes the command inside that container, then commits the container's filesystem changes as a new image layer. This new layer contains all changes made by the command, including added, modified, or deleted files. Docker stores these layers as immutable snapshots, enabling reuse and sharing.

Why designed this way?

This layered approach was designed to optimize storage and build speed by reusing unchanged layers. It also allows incremental builds and easy rollback. Alternatives like rebuilding entire images from scratch were slower and less efficient. The tradeoff is that each RUN creates a new layer, so combining commands is recommended to reduce image size.

Docker RUN Instruction Flow:

┌─────────────┐
│ Current    │
│ Image      │
└─────┬───────┘
      │
      ▼
┌─────────────┐
│ Create      │
│ Temp       │
│ Container  │
└─────┬───────┘
      │ Execute RUN command
      ▼
┌─────────────┐
│ Commit     │
│ Container  │
│ Changes    │
└─────┬───────┘
      │
      ▼
┌─────────────┐
│ New Image  │
│ Layer      │
└─────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does deleting files in one RUN command remove them from the final image? Commit yes or no.

Common Belief: If you delete files in a RUN command, they are removed from the final image.

Reality: Deleting files in one RUN command removes them only in that layer, but if they were added in a previous layer, the image size still includes them.

Why it matters: This misconception leads to unnecessarily large images because temporary files persist in earlier layers.

Quick: Do RUN commands execute when you start a container? Commit yes or no.

Common Belief: RUN commands run every time you start a container.

Reality: RUN commands execute only during image build, not when containers start.

Why it matters: Confusing RUN with container startup commands causes errors in container behavior and debugging.

Quick: Does using exec form RUN always require a shell? Commit yes or no.

Common Belief: RUN exec form runs commands inside a shell like the shell form.

Reality: RUN exec form runs commands directly without a shell, so shell features like variable expansion don't work.

Why it matters: Misunderstanding this causes commands to fail or behave unexpectedly.

Quick: Does changing a RUN command only rebuild that layer? Commit yes or no.

Common Belief: Changing one RUN command rebuilds only that layer.

Reality: Changing a RUN command invalidates its layer and all layers after it, causing a rebuild of subsequent layers.

Why it matters: This affects build times and caching strategies in complex Dockerfiles.

Expert Zone

1

RUN commands that rely on external resources (like package repositories) can cause cache invalidation if those resources change, even if the command text is unchanged.

2

Combining RUN commands with logical operators (&&) not only reduces layers but also ensures that subsequent commands run only if previous ones succeed, improving build reliability.

3

Using exec form RUN can improve security by avoiding shell injection risks and making command parsing more predictable.

When NOT to use

RUN should not be used for commands that need to run every time a container starts; use CMD or ENTRYPOINT instead. Also, avoid RUN for dynamic configuration that depends on runtime environment variables; use scripts or entrypoint logic for that.

Production Patterns

In production, RUN is used to install dependencies, set up environment tools, and prepare application code. Multi-stage builds often use RUN to build artifacts in one stage and copy them to a smaller final image. RUN commands are carefully ordered and combined to optimize caching and minimize image size.

Connections

Layered File Systems

RUN creates new layers in Docker's layered filesystem.

Understanding layered filesystems helps grasp why RUN commands add image size and how Docker reuses unchanged layers.

Continuous Integration (CI) Pipelines

RUN commands automate environment setup similar to CI build steps.

Knowing RUN parallels CI build scripts clarifies how container images are reproducible and consistent.

Software Build Systems

RUN commands are like build steps that prepare software environments.

Seeing RUN as build steps helps understand caching, dependencies, and incremental builds in Docker.

Common Pitfalls

#1 Creating many RUN commands without combining them.

Wrong approach:

RUN apt-get update
RUN apt-get install -y curl
RUN apt-get clean

Correct approach:

RUN apt-get update && apt-get install -y curl && apt-get clean

Root cause:

Not realizing each RUN creates a new layer, leading to larger images and slower builds.

#2 Deleting files in a separate RUN command after creating them.

Wrong approach:

RUN apt-get update
RUN rm -rf /var/lib/apt/lists/*

Correct approach:

RUN apt-get update && rm -rf /var/lib/apt/lists/*

Root cause:

Misunderstanding that layers are immutable and deleting files in a later layer doesn't remove them from earlier layers.

#3 Using shell features in exec form RUN commands.

Wrong approach:

RUN ["sh", "-c", "echo $HOME"]

Correct approach:

RUN echo $HOME

Root cause:

Not knowing exec form bypasses shell, so shell features like variable expansion don't work unless explicitly invoked.

Key Takeaways

RUN executes commands during image build to prepare the container environment before runtime.

Each RUN creates a new image layer, so combining commands reduces image size and speeds up builds.

RUN supports shell and exec forms; choosing the right one affects command behavior and security.

Understanding Docker's layer caching with RUN commands is essential for efficient and fast image builds.

RUN is for build-time setup; runtime commands belong in CMD or ENTRYPOINT.