Multi-stage builds for smaller images in MLOps - Step-by-Step Execution

Process Flow - Multi-stage builds for smaller images

Start: Define first stage

↓

Build app in first stage

↓

Define second stage

↓

Copy only needed files from first stage

↓

Build final smaller image

↓

Use smaller image for deployment

Multi-stage builds use multiple steps in one Dockerfile to build and then copy only needed files, making the final image smaller.

Execution Sample

MLOps

FROM node:18 AS build
WORKDIR /app
COPY package.json .
RUN npm install
COPY . .
RUN npm run build

FROM node:18-slim
WORKDIR /app
COPY --from=build /app/dist ./dist
CMD ["node", "dist/index.js"]

This Dockerfile builds the app in the first stage, then copies only the built files to a smaller final image.

Process Table

Step	Action	Stage	Files Present	Image Size Impact
1	Start first stage with full node image	build	All source files copied	Large size due to full dependencies
2	Run npm install and build	build	Node modules and build output created	Size grows due to dependencies and build artifacts
3	Start second stage with slim node image	final	Empty except base image files	Small base size
4	Copy only /app/dist from build stage	final	Only built files present	Size small, no dev dependencies
5	Set command to run built app	final	Ready to run	Final image optimized
6	Image ready for deployment	final	Contains only runtime files	Small image size achieved

💡 Final image contains only necessary runtime files, making it smaller than the build stage image.

Status Tracker

Variable	Start	After Step 2	After Step 4	Final
Files in build stage	source files only	source + node_modules + build output	N/A	N/A
Files in final stage	empty	empty	only /app/dist copied	only /app/dist present
Image size	base size	large due to dependencies	base slim size	small optimized size

Key Moments - 3 Insights

Why do we use two FROM statements in one Dockerfile?

The first FROM builds the full app with all tools, the second FROM starts a clean smaller image to copy only needed files, as shown in execution_table steps 1 and 3.

What happens if we copy all files from the build stage instead of just the build output?

The final image will be large because it includes unnecessary source and node_modules files, losing the size benefit shown in step 4.

Why is the final image smaller even though it runs the same app?

Because it contains only the built app files without development dependencies, as seen in the variable_tracker for final stage files.

Visual Quiz - 3 Questions

Look at the execution_table at step 4, what files are copied to the final image?

A Node modules only

B All source files

C /app/dist only

D Empty folder

💡 Hint

Check the 'Files Present' column at step 4 in the execution_table.

At which step does the image size become small and optimized?

A Step 6

B Step 2

C Step 5

D Step 4

💡 Hint

Look at the 'Image Size Impact' column and find when the final image is ready.

If we remove the second FROM line, how would the final image size change?

A It would be smaller

B It would be larger

C It would be the same

D It would fail to build

💡 Hint

Refer to the concept_flow and execution_table steps showing the purpose of the second stage.

Concept Snapshot

Multi-stage builds use multiple FROM statements in one Dockerfile.
First stage builds app with all dependencies.
Second stage copies only needed files to a smaller base image.
Result: smaller final image with just runtime files.
Use COPY --from=build to transfer files between stages.

Full Transcript

Multi-stage builds help create smaller Docker images by splitting the build process into stages. The first stage uses a full image to build the app with all dependencies. The second stage starts from a smaller base image and copies only the built app files from the first stage. This reduces the final image size by excluding development tools and source files. The Dockerfile uses multiple FROM statements to define stages and COPY --from to transfer files. This method is common in MLOps to optimize container sizes for deployment.

Practice

1. What is the main benefit of using multi-stage builds in Docker?

easy

A. They enable Docker images to run on any operating system without modification.

B. They create smaller and cleaner Docker images by separating build and runtime stages.

C. They automatically update the base image to the latest version.

D. They allow running multiple containers simultaneously.

Solution

1. Step 1: Understand multi-stage build concept

   Multi-stage builds separate the build environment from the runtime environment in Dockerfiles.

2. Step 2: Identify the benefit of separation

   This separation removes unnecessary build tools from the final image, making it smaller and cleaner.

3. Final Answer:

   They create smaller and cleaner Docker images by separating build and runtime stages. -> Option B

4. Quick Check:

   Multi-stage builds = smaller images [OK]

Hint: Multi-stage builds reduce image size by splitting build and runtime [OK]

Common Mistakes:

• Confusing multi-stage builds with running multiple containers
• Thinking multi-stage builds update base images automatically
• Assuming multi-stage builds change OS compatibility

2. Which of the following is the correct syntax to start a new stage named 'builder' in a Dockerfile?

easy

A. FROM ubuntu AS builder

B. STAGE builder FROM ubuntu

C. NEW STAGE builder FROM ubuntu

D. BUILD STAGE builder FROM ubuntu

Solution

1. Step 1: Recall Dockerfile multi-stage syntax

   To start a new build stage, Dockerfile uses 'FROM <image> AS <name>'.

2. Step 2: Match correct syntax

   Only 'FROM ubuntu AS builder' matches the correct syntax for naming a stage.

3. Final Answer:

   FROM ubuntu AS builder -> Option A

4. Quick Check:

   Stage naming uses 'FROM ... AS ...' [OK]

Hint: Use 'FROM image AS name' to start a new build stage [OK]

Common Mistakes:

• Using 'STAGE' keyword which does not exist
• Writing 'NEW STAGE' instead of 'FROM ... AS ...'
• Confusing 'BUILD STAGE' with Dockerfile syntax

3. Given this Dockerfile snippet, what will be the size effect on the final image?

FROM golang:1.20 AS builder
WORKDIR /app
COPY . .
RUN go build -o myapp

FROM alpine:latest
COPY --from=builder /app/myapp /usr/local/bin/myapp
CMD ["myapp"]

medium

A. The final image will fail to build due to missing Go compiler in the second stage.

B. The final image will be large because it includes the entire Go build environment.

C. The final image will be small because it only copies the built binary from the builder stage.

D. The final image will include both Alpine and Go base images merged.

Solution

1. Step 1: Analyze multi-stage build steps

   The first stage builds the Go binary using the full Go environment. The second stage uses a minimal Alpine image.

2. Step 2: Understand what is copied to final image

   Only the compiled binary '/app/myapp' is copied from the builder stage to the final image, excluding build tools.

3. Final Answer:

   The final image will be small because it only copies the built binary from the builder stage. -> Option C

4. Quick Check:

   Copying only binary = smaller final image [OK]

Hint: Final image size shrinks by copying only needed files [OK]

Common Mistakes:

• Assuming the entire build environment is included in final image
• Thinking the build fails due to missing compiler in second stage
• Believing base images merge into one large image

4. Identify the error in this Dockerfile snippet using multi-stage build:

FROM node:18 AS builder
WORKDIR /app
COPY package.json .
RUN npm install
COPY . .
RUN npm run build

FROM node:18
COPY --from=builder /app/dist ./dist
CMD ["node", "./dist/server.js"]

medium

A. The COPY command in the second stage has incorrect source path syntax.

B. The first stage is missing a WORKDIR declaration.

C. The CMD command is missing square brackets for JSON array syntax.

D. The second stage should use a smaller base image like 'node:18-alpine' to reduce size.

Solution

1. Step 1: Review base images used in both stages

   Both stages use 'node:18', which is a full Node image including build tools.

2. Step 2: Suggest optimization for smaller final image

   Using a smaller base like 'node:18-alpine' in the second stage reduces image size by excluding unnecessary tools.

3. Final Answer:

   The second stage should use a smaller base image like 'node:18-alpine' to reduce size. -> Option D

4. Quick Check:

   Use lightweight base images in final stage [OK]

Hint: Use lightweight base images in final stage for smaller images [OK]

Common Mistakes:

• Thinking COPY syntax is incorrect when it is valid
• Believing CMD needs different syntax here
• Assuming WORKDIR is missing in first stage

5. You want to build a Python app with dependencies installed only during build, but keep the final image minimal. Which multi-stage Dockerfile snippet achieves this best?

hard

A. FROM python:3.12 AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .

FROM python:3.12-slim
COPY --from=builder /app /app
CMD ["python", "/app/app.py"]

B. FROM python:3.12
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "/app/app.py"]

C. FROM python:3.12 AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "/app/app.py"]

D. FROM python:3.12-slim
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["python", "/app/app.py"]

Solution

1. Step 1: Understand requirement for minimal final image

   Dependencies should be installed in a build stage, not in the final image, to keep it small.

2. Step 2: Analyze options for multi-stage usage

   FROM python:3.12 AS builder
   WORKDIR /app
   COPY requirements.txt .
   RUN pip install -r requirements.txt
   COPY . .
   
   FROM python:3.12-slim
   COPY --from=builder /app /app
   CMD ["python", "/app/app.py"]

   uses a builder stage to install dependencies and copies only the app to a slim final image, achieving minimal size.

3. Final Answer:

   Option A correctly uses multi-stage build to keep final image minimal. -> Option A

4. Quick Check:

   Install dependencies in builder, copy to slim final image [OK]

Hint: Install dependencies in builder stage, copy only needed files to slim image [OK]

Common Mistakes:

• Installing dependencies directly in final image increasing size
• Not using multi-stage build at all
• Running app in builder stage instead of final stage