Why multi-stage builds reduce image size in Docker - Why It Works This Way

Overview - Why multi-stage builds reduce image size

What is it?

Multi-stage builds in Docker let you use multiple steps to create an image. Each step can use a different base image and only copy the needed parts to the final image. This way, you avoid keeping unnecessary files and tools in the final image. It helps make Docker images smaller and cleaner.

Why it matters

Without multi-stage builds, Docker images often include extra tools and files used only during building, making them large and slow to download or start. Smaller images save storage, speed up deployment, and reduce security risks by having fewer components. Multi-stage builds solve this by separating build and runtime environments.

Where it fits

Before learning multi-stage builds, you should understand basic Dockerfile syntax and how Docker images are built. After mastering this, you can explore advanced Docker optimizations, container security, and continuous integration pipelines that use efficient images.

Mental Model

Core Idea

Multi-stage builds let you build in steps and copy only what you need to the final image, cutting out all the extra stuff.

Think of it like...

It's like cooking a meal in a kitchen with many rooms: you prepare ingredients in one room with all your tools, then only bring the finished dish to the dining table, leaving the messy kitchen behind.

┌───────────────┐      ┌───────────────┐      ┌───────────────┐
│ Build Stage 1 │─────▶│ Build Stage 2 │─────▶│ Final Image   │
│ (with tools)  │      │ (compile app) │      │ (only needed) │
└───────────────┘      └───────────────┘      └───────────────┘

Build-Up - 6 Steps

1

Foundation

Understanding Docker Images and Layers

Concept: Docker images are made of layers stacked on each other, each adding files or changes.

A Docker image starts from a base image like Ubuntu. Each command in a Dockerfile adds a new layer. Layers include files, environment variables, or installed software. When you build an image, all layers combine to form the final image.

Result

You get a layered image that contains everything needed to run your app.

Understanding layers helps you see why extra files or tools in early steps stay in the final image unless removed.

2

Foundation

Basic Dockerfile Build Process

Concept: Docker builds images by running commands in order, keeping all files created in each step.

A Dockerfile might start with 'FROM ubuntu', then 'RUN apt-get install build-tools', then 'COPY source /app'. Each step adds files or software. The final image includes all these layers, even if some tools are only needed during build.

Result

The final image contains both build tools and app files, making it larger.

Knowing this shows why images can be unnecessarily big if build tools stay inside.

3

Intermediate

Introducing Multi-Stage Builds

🤔Before reading on: do you think copying files from one build stage to another can reduce image size? Commit to yes or no.

Concept: Multi-stage builds let you use multiple FROM statements and selectively copy files between stages.

You start with a build stage that has all tools to compile your app. Then you create a second stage with a minimal base image. Using 'COPY --from=build-stage', you copy only the compiled app files, leaving build tools behind.

Result

The final image is smaller because it contains only the app, not the build tools.

Knowing you can separate build and runtime environments lets you keep images lean and secure.

4

Intermediate

How COPY --from Works in Multi-Stage Builds

🤔Before reading on: does COPY --from copy the entire previous image or only specified files? Commit to your answer.

Concept: COPY --from copies only specified files or directories from a previous build stage, not the whole image.

In the Dockerfile, you name a build stage like 'AS builder'. Later, 'COPY --from=builder /app/bin /app/bin' copies only the compiled binaries. This avoids copying unnecessary files like source code or build caches.

Result

You control exactly what goes into the final image, reducing size and clutter.

Understanding selective copying prevents bloated images and improves build efficiency.

5

Advanced

Optimizing Multi-Stage Builds for Size

🤔Before reading on: do you think removing files in the build stage reduces final image size? Commit yes or no.

Concept: Cleaning up unnecessary files in build stages and copying only needed artifacts further reduces image size.

In the build stage, you can delete temporary files or caches before copying artifacts. Also, choosing minimal base images for the final stage (like alpine) shrinks the image. Combining these practices leads to very small, efficient images.

Result

Final images are minimal, fast to download, and secure.

Knowing how to clean build stages and pick minimal bases maximizes the benefits of multi-stage builds.

6

Expert

Surprising Effects of Layer Caching in Multi-Stage Builds

🤔Before reading on: do you think multi-stage builds always speed up rebuilds? Commit yes or no.

Concept: Layer caching can behave unexpectedly in multi-stage builds, affecting build speed and size.

Docker caches layers by command and context. If a build stage changes, later stages may rebuild too. Also, copying large files between stages can slow builds. Understanding cache invalidation and ordering commands carefully helps optimize build times and image size.

Result

You get faster builds and smaller images by managing cache and stage order well.

Knowing Docker's cache mechanics prevents slow builds and bloated images even with multi-stage builds.

Under the Hood

Docker builds images by executing each Dockerfile command in order, creating a new layer for each. Multi-stage builds use multiple FROM commands to create separate build environments. The COPY --from instruction copies files from one stage's filesystem to another without including that stage's entire image layers. This means only selected files are included in the final image layers, excluding build tools and temporary files.

Why designed this way?

Multi-stage builds were introduced to solve the problem of large images caused by including build dependencies. Before, developers had to manually clean images or use complex scripts. The design allows clear separation of build and runtime, improving security, size, and maintainability. Alternatives like manual cleanup were error-prone and less efficient.

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│ Build Stage   │       │ Copy Artifacts│       │ Final Image   │
│ (with tools)  │──────▶│ --from=build  │──────▶│ (minimal only)│
│ ┌───────────┐ │       │               │       │               │
│ │ Compile   │ │       │               │       │               │
│ │ App files │ │       │               │       │               │
│ └───────────┘ │       │               │       │               │
└───────────────┘       └───────────────┘       └───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does multi-stage build copy the entire previous stage by default? Commit yes or no.

Common Belief: Multi-stage builds copy the whole previous image stage automatically to the final image.

Reality: Only files explicitly copied with COPY --from are included; the rest of the previous stage is excluded.

Why it matters: Assuming the whole stage is copied leads to confusion about image size and wasted effort trying to reduce size.

Quick: Does removing build tools in the build stage always reduce final image size? Commit yes or no.

Common Belief: Deleting build tools inside the build stage reduces the final image size.

Reality: Removing files in build stages does not reduce final image size unless those files are excluded from the final stage via COPY --from.

Why it matters: Misunderstanding this causes wasted cleanup steps that don't affect the final image size.

Quick: Do multi-stage builds always speed up Docker build times? Commit yes or no.

Common Belief: Multi-stage builds always make Docker builds faster.

Reality: Multi-stage builds can sometimes slow builds due to cache invalidation and copying large files between stages.

Why it matters: Expecting faster builds without managing cache and stage order can lead to frustration and inefficient workflows.

Quick: Is multi-stage build the only way to reduce image size? Commit yes or no.

Common Belief: Multi-stage builds are the only method to reduce Docker image size.

Reality: Other methods like image slimming tools, minimal base images, and manual cleanup also reduce size.

Why it matters: Relying solely on multi-stage builds may miss other optimization opportunities.

Expert Zone

1

Multi-stage builds can be combined with build cache strategies to optimize CI/CD pipeline speed and resource use.

2

Choosing the right base image for each stage affects both build time and final image size significantly.

3

Copying only necessary files with precise paths avoids accidentally including sensitive or bulky files.

When NOT to use

Avoid multi-stage builds when your application is extremely simple and does not require build tools, or when build time is more critical than image size. Alternatives include single-stage builds with manual cleanup or using pre-built minimal base images.

Production Patterns

In production, multi-stage builds are used to separate compilation and packaging from runtime. For example, compiling Go or Java apps in one stage, then copying only binaries to a minimal Alpine image. This pattern reduces attack surface and speeds up deployment.

Connections

Continuous Integration Pipelines

Multi-stage builds integrate with CI pipelines by producing small, efficient images for deployment.

Understanding multi-stage builds helps optimize CI workflows by reducing build times and artifact sizes.

Software Build Systems

Multi-stage builds mimic traditional build systems that separate compile and package steps.

Knowing build systems clarifies why separating build and runtime environments reduces complexity and size.

Lean Manufacturing

Both focus on eliminating waste and only delivering what is needed.

Recognizing this connection shows how principles from manufacturing apply to software delivery efficiency.

Common Pitfalls

#1 Including build tools in the final image by copying entire build stage.

Wrong approach:

FROM golang:1.20 AS builder
WORKDIR /app
COPY . .
RUN go build -o myapp
FROM alpine:3.18
COPY --from=builder /app /app
CMD ["/app/myapp"]

Correct approach:

FROM golang:1.20 AS builder
WORKDIR /app
COPY . .
RUN go build -o myapp
FROM alpine:3.18
COPY --from=builder /app/myapp /app/myapp
CMD ["/app/myapp"]

Root cause:

Copying the whole /app directory includes source code and build tools, bloating the final image.

#2 Trying to reduce image size by deleting files in build stage without excluding them.

Wrong approach:

FROM node:18 AS builder
WORKDIR /app
COPY package.json .
RUN npm install
RUN rm -rf /app/node_modules/.cache
COPY . .
RUN npm run build
FROM node:18
COPY --from=builder /app /app
CMD ["node", "/app/index.js"]

Correct approach:

FROM node:18 AS builder
WORKDIR /app
COPY package.json .
RUN npm install
COPY . .
RUN npm run build
FROM node:18
COPY --from=builder /app/dist /app
CMD ["node", "/app/index.js"]

Root cause:

Deleting files in builder stage doesn't reduce final image size if the entire /app folder is copied.

#3 Assuming multi-stage builds always speed up builds without managing cache.

Wrong approach:

FROM python:3.11 AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
FROM python:3.11
COPY --from=builder /app /app
CMD ["python", "/app/app.py"]

Correct approach:

FROM python:3.11 AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
FROM python:3.11
COPY --from=builder /app/app.py /app/app.py
COPY --from=builder /app/venv /app/venv
CMD ["python", "/app/app.py"]

Root cause:

Copying entire /app folder invalidates cache and slows builds; selective copying and cache management improve speed.

Key Takeaways

Multi-stage builds let you separate build and runtime environments to keep Docker images small and clean.

Only files explicitly copied from build stages appear in the final image, preventing unnecessary bloat.

Understanding Docker layers and caching is key to optimizing build speed and image size with multi-stage builds.

Multi-stage builds improve security by excluding build tools and reduce deployment time with smaller images.

Effective use of multi-stage builds requires careful selection of base images and precise copying of artifacts.