Multi-stage for different environments in Docker - Deep Dive

Overview - Multi-stage for different environments

What is it?

Multi-stage builds in Docker let you create a single Dockerfile with multiple steps, each producing parts of the final image. This helps you build different versions of your app for various environments like development, testing, and production without repeating code. Each stage can use different tools or settings, and only the needed parts are kept in the final image. This makes images smaller and more secure.

Why it matters

Without multi-stage builds, you would need separate Dockerfiles or large images containing all build tools and files, making deployment slow and error-prone. Multi-stage builds solve this by letting you share common steps and produce optimized images for each environment, saving time, space, and reducing bugs. This improves developer productivity and system reliability.

Where it fits

Before learning multi-stage builds, you should understand basic Dockerfiles and image layering. After mastering this, you can explore advanced Docker optimizations, CI/CD pipelines using Docker, and environment-specific deployment strategies.

Mental Model

Core Idea

Multi-stage builds let you combine multiple build steps in one Dockerfile to create optimized images tailored for different environments.

Think of it like...

It's like cooking a meal in stages: you prepare ingredients in one kitchen, then only bring the finished parts to the dining table, leaving the mess and tools behind.

┌───────────────┐      ┌───────────────┐      ┌───────────────┐
│ Build Stage 1 │─────▶│ Build Stage 2 │─────▶│ Final Image   │
│ (e.g. compile)│      │ (e.g. test)   │      │ (only needed  │
│               │      │               │      │ files kept)   │
└───────────────┘      └───────────────┘      └───────────────┘

Build-Up - 6 Steps

1

Foundation

Understanding basic Dockerfile stages

Concept: Learn what a Dockerfile stage is and how Docker builds images step-by-step.

A Dockerfile is a list of instructions to build an image. Each instruction creates a layer. Normally, a Dockerfile has one main stage. You write commands like FROM, RUN, COPY, and CMD. The FROM command starts a new stage with a base image. Without multi-stage, you have one FROM and build everything in one image.

Result

You get a Docker image with all files and layers from the single stage.

Understanding that each FROM starts a new stage is key to grasping how multi-stage builds separate concerns.

2

Foundation

Why separate build and runtime stages

Concept: Learn why it's useful to have one stage for building code and another for running it.

Build tools like compilers or package managers are big and not needed at runtime. By separating build and runtime stages, you keep the final image small and secure. For example, you can compile your app in one stage and copy only the compiled files to the final stage.

Result

Final image is smaller and contains only what is needed to run the app.

Knowing that build tools can be left out of the final image helps optimize size and security.

3

Intermediate

Using named stages for environment targets

🤔Before reading on: do you think you can name stages and selectively use them later? Commit to your answer.

Concept: You can name each stage and choose which one to build or copy from, enabling environment-specific images.

In Dockerfile, you write FROM base AS builder to name a stage. Later stages can COPY --from=builder to get files. You can build a specific stage with docker build --target=builder. This lets you create stages for dev, test, and prod in one file.

Result

You can build different images from the same Dockerfile by targeting stages.

Understanding named stages unlocks flexible builds for multiple environments without duplicating Dockerfiles.

4

Intermediate

Sharing common steps across stages

🤔Before reading on: do you think each stage must repeat all commands or can they share steps? Commit to your answer.

Concept: Stages can build on each other by using previous stages as base images, sharing common setup steps.

You can start a stage FROM a previous stage's name instead of a base image. This lets you reuse installed dependencies or compiled files. For example, dev and prod stages can both start FROM builder stage to avoid repeating build steps.

Result

Build process is faster and Dockerfile is cleaner with shared steps.

Knowing how to reuse stages prevents duplication and speeds up builds.

5

Advanced

Optimizing multi-stage builds for production

🤔Before reading on: do you think the final production image should include build tools? Commit to your answer.

Concept: The production stage should be minimal, copying only runtime essentials and excluding build tools and source code.

In the final stage, use a lightweight base like alpine or scratch. COPY only compiled binaries or necessary files from build stages. Avoid installing extra packages. This reduces image size and attack surface. Use multi-stage to strip debug info or run tests in earlier stages but exclude them in production.

Result

Production images are small, secure, and fast to deploy.

Understanding minimal final images improves security and performance in real deployments.

6

Expert

Advanced tricks with multi-stage builds

🤔Before reading on: can multi-stage builds help with caching and conditional builds? Commit to your answer.

Concept: Multi-stage builds can leverage Docker cache effectively and conditionally include files or steps based on build arguments.

You can use ARG and conditional COPY or RUN commands to customize stages. Docker caches layers per stage, so changing one stage doesn't rebuild all. You can also create stages for testing, linting, or debugging that don't affect production. This makes builds faster and more flexible.

Result

Builds become efficient, customizable, and maintainable in complex projects.

Knowing how to use build arguments and caching with multi-stage builds unlocks powerful CI/CD optimizations.

Under the Hood

Docker processes a Dockerfile line by line, creating layers for each instruction. When it encounters FROM, it starts a new build stage with a fresh base image. Each stage builds its own layers independently. COPY --from= allows copying files from previous stages by referencing their names or indexes. Docker caches layers per stage, so unchanged steps are reused. The final image is the last stage built or the stage specified by --target.

Why designed this way?

Multi-stage builds were introduced to solve the problem of bloated images containing build tools and source code. Before, developers had to maintain multiple Dockerfiles or manually clean images. This design allows sharing build logic in one file, reduces duplication, and produces optimized images. Alternatives like separate Dockerfiles were harder to maintain and error-prone.

Dockerfile
  ├─ Stage 1: FROM base AS builder
  │    ├─ RUN compile code
  │    └─ OUTPUT compiled files
  ├─ Stage 2: FROM base AS tester
  │    ├─ COPY --from=builder compiled files
  │    └─ RUN tests
  └─ Stage 3: FROM lightweight base AS final
       ├─ COPY --from=builder compiled files
       └─ CMD run app

Myth Busters - 4 Common Misconceptions

Quick: Does multi-stage build always produce multiple images? Commit yes or no.

Common Belief: Multi-stage builds create multiple separate images, one per stage.

Reality: Multi-stage builds produce only one final image per build, usually the last or targeted stage. Intermediate stages are temporary and not saved as separate images unless explicitly tagged.

Why it matters: Thinking multiple images are created can confuse image management and lead to unnecessary cleanup or tagging.

Quick: Can you copy files from any stage regardless of order? Commit yes or no.

Common Belief: You can copy files from any stage in the Dockerfile, even if it appears later.

Reality: You can only copy files from stages defined before the current stage. Later stages are not accessible for copying.

Why it matters: Misunderstanding this causes build errors and confusion about stage dependencies.

Quick: Is it safe to include build tools in production images for debugging? Commit yes or no.

Common Belief: Including build tools in production images is fine and helps debugging.

Reality: Including build tools increases image size and attack surface, reducing security and performance. Debugging should be done in separate images or with remote tools.

Why it matters: Ignoring this leads to bloated, vulnerable production images.

Quick: Does multi-stage build automatically reduce image size? Commit yes or no.

Common Belief: Using multi-stage builds always results in smaller images.

Reality: Multi-stage builds enable smaller images, but only if you carefully copy only needed files and use minimal base images. Poorly designed stages can still produce large images.

Why it matters: Assuming automatic optimization can cause neglect of best practices and large images.

Expert Zone

1

Stages can be reused as base images in other Dockerfiles by tagging intermediate images, enabling modular builds.

2

Build cache is scoped per stage, so changing one stage invalidates only that stage's cache, speeding up incremental builds.

3

Using multi-stage builds with build arguments allows conditional inclusion of files or steps, enabling dynamic environment customization.

When NOT to use

Multi-stage builds are less useful for very simple images or when build steps are minimal. In such cases, a single-stage Dockerfile is simpler. Also, if you need to debug inside the image frequently, a development image with build tools included might be better. Alternatives include separate Dockerfiles per environment or external build scripts.

Production Patterns

In production, multi-stage builds are used to compile code in a builder stage, run tests in a test stage, and produce a minimal runtime image with only compiled binaries and runtime dependencies. CI/CD pipelines build and push only the final stage image. Some teams use multi-stage builds to embed version info or perform security scans in intermediate stages.

Connections

Continuous Integration/Continuous Deployment (CI/CD)

Multi-stage builds integrate tightly with CI/CD pipelines by producing environment-specific images in one build process.

Understanding multi-stage builds helps optimize CI/CD workflows by reducing build times and managing environment differences efficiently.

Software Build Systems

Multi-stage builds mirror concepts in build systems where compilation, testing, and packaging are separate steps.

Knowing build system stages clarifies why separating build and runtime in Docker improves maintainability and performance.

Manufacturing Assembly Lines

Multi-stage builds are like assembly lines where parts are prepared in stages before final assembly.

Seeing Docker builds as assembly lines helps grasp the efficiency and modularity benefits of multi-stage builds.

Common Pitfalls

#1 Copying unnecessary files from build stages to final image.

Wrong approach:

COPY --from=builder / /app

Correct approach:

COPY --from=builder /app/build /app

Root cause:

Not specifying precise paths causes large images and leaks build artifacts.

#2 Using heavy base images in final stage.

Wrong approach:

FROM ubuntu:20.04 AS final
COPY --from=builder /app /app

Correct approach:

FROM alpine:3.18 AS final
COPY --from=builder /app /app

Root cause:

Choosing large base images increases image size and attack surface unnecessarily.

#3 Referencing a stage that appears later in the Dockerfile.

Wrong approach:

COPY --from=final /app /app

Correct approach:

COPY --from=builder /app /app

Root cause:

Docker processes stages top-down; later stages are not accessible for copying.

Key Takeaways

Multi-stage builds let you create optimized Docker images for different environments in one file by separating build steps.

Naming stages and using COPY --from= allows selective reuse of files and tools, reducing duplication and image size.

Final production images should be minimal, containing only runtime essentials to improve security and performance.

Understanding Docker's build cache and stage order is crucial to avoid common errors and speed up builds.

Multi-stage builds integrate well with CI/CD and mirror concepts from software build systems and manufacturing for efficient workflows.