Multi-stage builds for smaller images in MLOps - Mini Project: Build & Apply

Multi-stage builds for smaller images

📖 Scenario: You are working on a machine learning project that requires a Docker image to run your model training script. You want to keep the final Docker image small to save storage and speed up deployment.Multi-stage builds in Docker help you do this by letting you use one image to build your code and another smaller image to run it.

🎯 Goal: Build a Dockerfile using multi-stage builds that first installs all dependencies and copies your training script, then creates a smaller final image that only contains the necessary runtime files.

📋 What You'll Learn

Create a first build stage named builder using the python:3.12-slim image

In the builder stage, copy train.py and install scikit-learn

Create a second stage named runtime using the python:3.12-alpine image

Copy only the train.py file from the builder stage to the runtime stage

Set the default command to run python train.py in the runtime stage

💡 Why This Matters

🌍 Real World

Multi-stage builds are used in real projects to reduce Docker image size, which saves bandwidth and speeds up deployment.

💼 Career

Understanding multi-stage builds is important for DevOps and MLOps roles to optimize containerized machine learning workflows.

1

Create the builder stage

Write a Dockerfile line to start a build stage named builder using the image python:3.12-slim.

MLOps

# Your code here

Hint

Use the FROM instruction with AS builder to name the stage.

2

Copy train.py and install dependencies in builder

In the builder stage, add lines to copy train.py into the image and install the scikit-learn package using pip.

MLOps

FROM python:3.12-slim AS builder
# Your code here

Hint

Use COPY train.py /app/train.py and RUN pip install scikit-learn.

3

Create the runtime stage and copy train.py

Start a new stage named runtime using python:3.12-alpine. Then copy train.py from the builder stage to /app/train.py in this stage.

MLOps

FROM python:3.12-slim AS builder
COPY train.py /app/train.py
RUN pip install scikit-learn
# Your code here

Hint

Use FROM python:3.12-alpine AS runtime and COPY --from=builder /app/train.py /app/train.py.

4

Set the default command to run train.py

In the runtime stage, add a line to set the default command to python /app/train.py using the CMD instruction.

MLOps

FROM python:3.12-slim AS builder
COPY train.py /app/train.py
RUN pip install scikit-learn

FROM python:3.12-alpine AS runtime
COPY --from=builder /app/train.py /app/train.py
# Your code here

Hint

Use CMD ["python", "/app/train.py"] to set the default command.

Practice

1. What is the main benefit of using multi-stage builds in Docker?

easy

A. They enable Docker images to run on any operating system without modification.

B. They create smaller and cleaner Docker images by separating build and runtime stages.

C. They automatically update the base image to the latest version.

D. They allow running multiple containers simultaneously.

Solution

1. Step 1: Understand multi-stage build concept

   Multi-stage builds separate the build environment from the runtime environment in Dockerfiles.

2. Step 2: Identify the benefit of separation

   This separation removes unnecessary build tools from the final image, making it smaller and cleaner.

3. Final Answer:

   They create smaller and cleaner Docker images by separating build and runtime stages. -> Option B

4. Quick Check:

   Multi-stage builds = smaller images [OK]

Hint: Multi-stage builds reduce image size by splitting build and runtime [OK]

Common Mistakes:

• Confusing multi-stage builds with running multiple containers
• Thinking multi-stage builds update base images automatically
• Assuming multi-stage builds change OS compatibility

2. Which of the following is the correct syntax to start a new stage named 'builder' in a Dockerfile?

easy

A. FROM ubuntu AS builder

B. STAGE builder FROM ubuntu

C. NEW STAGE builder FROM ubuntu

D. BUILD STAGE builder FROM ubuntu

Solution

1. Step 1: Recall Dockerfile multi-stage syntax

   To start a new build stage, Dockerfile uses 'FROM <image> AS <name>'.

2. Step 2: Match correct syntax

   Only 'FROM ubuntu AS builder' matches the correct syntax for naming a stage.

3. Final Answer:

   FROM ubuntu AS builder -> Option A

4. Quick Check:

   Stage naming uses 'FROM ... AS ...' [OK]

Hint: Use 'FROM image AS name' to start a new build stage [OK]

Common Mistakes:

• Using 'STAGE' keyword which does not exist
• Writing 'NEW STAGE' instead of 'FROM ... AS ...'
• Confusing 'BUILD STAGE' with Dockerfile syntax

3. Given this Dockerfile snippet, what will be the size effect on the final image?

FROM golang:1.20 AS builder
WORKDIR /app
COPY . .
RUN go build -o myapp

FROM alpine:latest
COPY --from=builder /app/myapp /usr/local/bin/myapp
CMD ["myapp"]

medium

A. The final image will fail to build due to missing Go compiler in the second stage.

B. The final image will be large because it includes the entire Go build environment.

C. The final image will be small because it only copies the built binary from the builder stage.

D. The final image will include both Alpine and Go base images merged.

Solution

1. Step 1: Analyze multi-stage build steps

   The first stage builds the Go binary using the full Go environment. The second stage uses a minimal Alpine image.

2. Step 2: Understand what is copied to final image

   Only the compiled binary '/app/myapp' is copied from the builder stage to the final image, excluding build tools.

3. Final Answer:

   The final image will be small because it only copies the built binary from the builder stage. -> Option C

4. Quick Check:

   Copying only binary = smaller final image [OK]

Hint: Final image size shrinks by copying only needed files [OK]

Common Mistakes:

• Assuming the entire build environment is included in final image
• Thinking the build fails due to missing compiler in second stage
• Believing base images merge into one large image

4. Identify the error in this Dockerfile snippet using multi-stage build:

FROM node:18 AS builder
WORKDIR /app
COPY package.json .
RUN npm install
COPY . .
RUN npm run build

FROM node:18
COPY --from=builder /app/dist ./dist
CMD ["node", "./dist/server.js"]

medium

A. The COPY command in the second stage has incorrect source path syntax.

B. The first stage is missing a WORKDIR declaration.

C. The CMD command is missing square brackets for JSON array syntax.

D. The second stage should use a smaller base image like 'node:18-alpine' to reduce size.

Solution

1. Step 1: Review base images used in both stages

   Both stages use 'node:18', which is a full Node image including build tools.

2. Step 2: Suggest optimization for smaller final image

   Using a smaller base like 'node:18-alpine' in the second stage reduces image size by excluding unnecessary tools.

3. Final Answer:

   The second stage should use a smaller base image like 'node:18-alpine' to reduce size. -> Option D

4. Quick Check:

   Use lightweight base images in final stage [OK]

Hint: Use lightweight base images in final stage for smaller images [OK]

Common Mistakes:

• Thinking COPY syntax is incorrect when it is valid
• Believing CMD needs different syntax here
• Assuming WORKDIR is missing in first stage

5. You want to build a Python app with dependencies installed only during build, but keep the final image minimal. Which multi-stage Dockerfile snippet achieves this best?

hard

A. FROM python:3.12 AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .

FROM python:3.12-slim
COPY --from=builder /app /app
CMD ["python", "/app/app.py"]

B. FROM python:3.12
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "/app/app.py"]

C. FROM python:3.12 AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "/app/app.py"]

D. FROM python:3.12-slim
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["python", "/app/app.py"]

Solution

1. Step 1: Understand requirement for minimal final image

   Dependencies should be installed in a build stage, not in the final image, to keep it small.

2. Step 2: Analyze options for multi-stage usage

   FROM python:3.12 AS builder
   WORKDIR /app
   COPY requirements.txt .
   RUN pip install -r requirements.txt
   COPY . .
   
   FROM python:3.12-slim
   COPY --from=builder /app /app
   CMD ["python", "/app/app.py"]

   uses a builder stage to install dependencies and copies only the app to a slim final image, achieving minimal size.

3. Final Answer:

   Option A correctly uses multi-stage build to keep final image minimal. -> Option A

4. Quick Check:

   Install dependencies in builder, copy to slim final image [OK]

Hint: Install dependencies in builder stage, copy only needed files to slim image [OK]

Common Mistakes:

• Installing dependencies directly in final image increasing size
• Not using multi-stage build at all
• Running app in builder stage instead of final stage