Docker for ML reproducibility in MLOps - Step-by-Step Execution

Process Flow - Docker for ML reproducibility

Write Dockerfile

↓

Build Docker Image

↓

Run Container with ML Code

↓

ML Code Executes in Container

↓

Results Reproducible Anywhere

↓

Share Image or Dockerfile

This flow shows how you create a Docker image from a Dockerfile, run your ML code inside a container, and get reproducible results anywhere.

Execution Sample

MLOps

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . ./
CMD ["python", "train.py"]

This Dockerfile sets up a Python 3.12 environment, installs dependencies, copies ML code, and runs training.

Process Table

Step	Action	Details	Result
1	Read Dockerfile	FROM python:3.12-slim	Base image set to Python 3.12 slim
2	Set working directory	WORKDIR /app	Working directory inside container is /app
3	Copy requirements.txt	COPY requirements.txt ./	requirements.txt copied to /app
4	Install dependencies	RUN pip install -r requirements.txt	Python packages installed
5	Copy ML code	COPY . ./	All local files copied to /app
6	Set command	CMD ["python", "train.py"]	Container will run train.py on start
7	Build image	docker build -t ml-model .	Docker image 'ml-model' created
8	Run container	docker run ml-model	ML training starts inside container
9	Training completes	train.py finishes	Model trained reproducibly inside container

💡 Training completes and container stops; environment is consistent everywhere

Status Tracker

Variable	Start	After Step 3	After Step 4	After Step 5	After Step 8	Final
Docker Image	None	Base python:3.12-slim	With dependencies installed	With ML code copied	Image built and ready	Image used to run container
Container State	None	Not started	Not started	Not started	Running ML training	Training complete

Key Moments - 2 Insights

Why do we copy requirements.txt and run pip install separately before copying all files?

Because installing dependencies first uses Docker cache efficiently. If only code changes but dependencies don't, Docker skips reinstalling packages (see execution_table steps 3 and 4).

How does running ML code inside a container help with reproducibility?

The container has the exact same environment everywhere, so the ML code runs with the same Python version and packages (see execution_table steps 8 and 9).

Visual Quiz - 3 Questions

Look at the execution table, at which step is the Python environment prepared with required packages?

A Step 6

B Step 3

C Step 4

D Step 8

💡 Hint

Check the 'Action' and 'Details' columns for pip install command in step 4

At which step does the container start running the ML training code?

A Step 8

B Step 7

C Step 6

D Step 9

💡 Hint

Look for 'Run container' action in the execution table

If you change only train.py code, which step can Docker reuse from cache to save time?

A Step 5 (copy ML code)

B Step 4 (install dependencies)

C Step 7 (build image)

D Step 8 (run container)

💡 Hint

Refer to key moment about caching and see steps 3 and 4 in execution_table

Concept Snapshot

Docker for ML reproducibility:
- Write Dockerfile with base image, dependencies, and code
- Build image to create consistent environment
- Run container to execute ML code
- Results are reproducible anywhere with same image
- Use caching by separating dependency install from code copy

Full Transcript

Docker helps make machine learning reproducible by packaging code and environment together. First, you write a Dockerfile starting from a Python base image. Then you copy your requirements.txt and install dependencies. Next, you copy your ML code and set the command to run training. Building the image creates a snapshot of this environment. Running a container from this image executes your ML training inside a consistent setup. This means your results will be the same on any machine. Docker caching speeds up rebuilds by reusing steps if dependencies don't change. This step-by-step process ensures your ML work is easy to share and reproduce.

Practice

1. What is the main benefit of using Docker for machine learning projects?

easy

A. It replaces the need for writing ML code.

B. It automatically improves the accuracy of ML models.

C. It ensures the ML code runs the same way on any machine.

D. It speeds up the training process by using GPUs only.

Solution

1. Step 1: Understand Docker's purpose in ML

   Docker packages the code and environment so it runs identically anywhere.

2. Step 2: Compare options

   Only 'It ensures the ML code runs the same way on any machine.' describes this reproducibility benefit correctly.

3. Final Answer:

   It ensures the ML code runs the same way on any machine. -> Option C

4. Quick Check:

   Docker = consistent environment [OK]

Hint: Docker = same environment everywhere [OK]

Common Mistakes:

• Thinking Docker improves model accuracy automatically
• Believing Docker replaces ML coding
• Assuming Docker only speeds up training

2. Which of the following is the correct way to start a Docker container from an image named ml-image?

easy

A. docker run ml-image

B. docker start ml-image

C. docker build ml-image

D. docker create ml-image

Solution

1. Step 1: Identify the command to run a container

   The docker run command starts a container from an image.

2. Step 2: Understand other commands

   docker start starts stopped containers, docker build creates images, docker create creates containers but does not start them.

3. Final Answer:

   docker run ml-image -> Option A

4. Quick Check:

   Run container = docker run [OK]

Hint: Use 'docker run' to start containers from images [OK]

Common Mistakes:

• Using 'docker start' to run new containers
• Confusing 'docker build' with running containers
• Using 'docker create' without starting container

3. Given this Dockerfile snippet:

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . ./
CMD ["python", "train.py"]

What happens when you build and run this Docker image?

medium

A. The container installs dependencies and runs train.py automatically.

B. The container only copies files but does not install dependencies.

C. The container runs train.py without installing dependencies.

D. The container fails because requirements.txt is missing.

Solution

1. Step 1: Analyze Dockerfile steps

   The Dockerfile sets Python 3.12, copies requirements.txt, installs dependencies, copies code, then runs train.py.

2. Step 2: Understand build and run behavior

   Building installs dependencies; running executes train.py automatically as CMD defines the command.

3. Final Answer:

   The container installs dependencies and runs train.py automatically. -> Option A

4. Quick Check:

   Dockerfile CMD runs train.py after setup [OK]

Hint: CMD runs script after dependencies installed [OK]

Common Mistakes:

• Assuming dependencies are not installed
• Thinking CMD runs during build, not run
• Believing files are not copied before run

4. You wrote a Dockerfile but when running the container, your ML code fails with "ModuleNotFoundError". What is the most likely cause?

medium

A. You forgot to copy your code files into the image.

B. You did not expose the correct port in Dockerfile.

C. You used the wrong base image version.

D. You did not install the required Python packages.

Solution

1. Step 1: Understand ModuleNotFoundError meaning

   This error means Python cannot find a required package or module.

2. Step 2: Identify cause related to Dockerfile

   Not installing required packages (missing pip install) causes this error, not copying code or ports.

3. Final Answer:

   You did not install the required Python packages. -> Option D

4. Quick Check:

   ModuleNotFoundError = missing packages [OK]

Hint: Missing packages cause ModuleNotFoundError [OK]

Common Mistakes:

• Blaming missing code files instead of packages
• Confusing port exposure with module errors
• Assuming base image version always causes this

5. You want to ensure your ML training runs reproducibly with Docker, including specific Python version, dependencies, and data files. Which Dockerfile snippet best achieves this?

hard

A. FROM ubuntu:latest
RUN apt-get update
COPY train.py ./
CMD ["python", "train.py"]

B. FROM python:3.12
WORKDIR /app
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY data/ ./data/
COPY train.py ./
CMD ["python", "train.py"]

C. FROM python:latest
COPY train.py ./
CMD ["python", "train.py"]

D. FROM python:3.12
RUN pip install numpy
CMD ["python", "train.py"]

Solution

1. Step 1: Check for full environment setup

   FROM python:3.12 WORKDIR /app COPY requirements.txt ./ RUN pip install -r requirements.txt COPY data/ ./data/ COPY train.py ./ CMD ["python", "train.py"] sets Python 3.12, installs dependencies from requirements.txt, copies data and code, then runs training.

2. Step 2: Compare other options

   The other options miss dependencies, data files, or use generic Python versions, risking non-reproducibility.

3. Final Answer:

   FROM python:3.12 WORKDIR /app COPY requirements.txt ./ RUN pip install -r requirements.txt COPY data/ ./data/ COPY train.py ./ CMD ["python", "train.py"] -> Option B

4. Quick Check:

   Complete setup = reproducibility [OK]

Hint: Copy code, data, install deps, set Python version [OK]

Common Mistakes:

• Skipping dependency installation
• Not copying data files needed for training
• Using generic or latest Python versions