Docker for ML reproducibility in MLOps - Commands & Configuration

Introduction

Machine learning projects often need the same software setup to run correctly. Docker helps by packaging your ML code, libraries, and environment into one container. This makes sure your ML work runs the same way everywhere.

When you want to share your ML model with others and ensure it runs exactly the same on their computers.

When you need to run your ML training on different machines without worrying about software differences.

When you want to keep your ML environment clean and separate from other projects on your computer.

When you want to deploy your ML model to a server or cloud and be sure it works as tested.

When you want to save the exact setup of your ML experiment for future reuse or auditing.

Config File - Dockerfile

Dockerfile

FROM python:3.10-slim

WORKDIR /app

COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

COPY . ./

CMD ["python", "train.py"]

This Dockerfile starts from a small Python 3.10 image.

It sets the working folder inside the container to /app.

It copies the requirements.txt file and installs the Python packages listed there.

Then it copies all your ML code into the container.

Finally, it runs the training script train.py when the container starts.

Commands

This command builds a Docker image named 'ml-reproducible' with tag '1.0' from the Dockerfile in the current folder. It packages your ML environment and code.

Terminal

docker build -t ml-reproducible:1.0 .

Expected Output

Sending build context to Docker daemon  5.12MB

Step 1/6 : FROM python:3.10-slim

 ---> 123abc456def

Step 2/6 : WORKDIR /app

 ---> Using cache

 ---> 789def012abc

Step 3/6 : COPY requirements.txt ./

 ---> Using cache

 ---> 345ghi678jkl

Step 4/6 : RUN pip install --no-cache-dir -r requirements.txt

 ---> Running in abc123def456

Collecting numpy

Installing collected packages: numpy

Successfully installed numpy-1.24.2

Removing intermediate container abc123def456

 ---> 901mno234pqr

Step 5/6 : COPY . ./

 ---> 567stu890vwx

Step 6/6 : CMD ["python", "train.py"]

 ---> Running in def789ghi012

Removing intermediate container def789ghi012

 ---> 345yz012abc

Successfully built 345yz012abc

Successfully tagged ml-reproducible:1.0

→

-t - Assigns a name and tag to the image for easy reference

This command runs the Docker container from the image you built. It starts the ML training script inside the container. The --rm flag removes the container after it finishes.

Terminal

docker run --rm ml-reproducible:1.0

Expected Output

Training started...

Epoch 1/10

Loss: 0.45

Epoch 10/10

Loss: 0.05

Training complete.

→

--rm - Automatically removes the container after it stops to keep your system clean

This command lists all Docker images on your system so you can see the 'ml-reproducible:1.0' image you created.

Terminal

docker images

Expected Output

REPOSITORY          TAG       IMAGE ID       CREATED         SIZE

ml-reproducible     1.0       345yz012abc    2 minutes ago   150MB

Key Concept

If you remember nothing else from this pattern, remember: Docker packages your ML code and environment together so your work runs the same everywhere.

Common Mistakes

Not including all required files in the Docker image by missing COPY commands.

The container will not have your ML code or dependencies, so the training will fail.

Make sure to COPY all necessary files like your code and requirements.txt into the image.

Running the container without the --rm flag and leaving stopped containers behind.

This clutters your system with unused containers, wasting space.

Use --rm to automatically clean up containers after they finish.

Not specifying a tag when building the image, causing confusion with multiple images.

It becomes hard to know which image version you are running or updating.

Always use -t with a clear name and version tag like ml-reproducible:1.0.

Summary

Create a Dockerfile to define your ML environment and code setup.

Build a Docker image from the Dockerfile to package your ML project.

Run the Docker container to execute your ML training reproducibly.

Use docker images to verify your image is created and available.

Practice

1. What is the main benefit of using Docker for machine learning projects?

easy

A. It replaces the need for writing ML code.

B. It automatically improves the accuracy of ML models.

C. It ensures the ML code runs the same way on any machine.

D. It speeds up the training process by using GPUs only.

Solution

1. Step 1: Understand Docker's purpose in ML

   Docker packages the code and environment so it runs identically anywhere.

2. Step 2: Compare options

   Only 'It ensures the ML code runs the same way on any machine.' describes this reproducibility benefit correctly.

3. Final Answer:

   It ensures the ML code runs the same way on any machine. -> Option C

4. Quick Check:

   Docker = consistent environment [OK]

Hint: Docker = same environment everywhere [OK]

Common Mistakes:

• Thinking Docker improves model accuracy automatically
• Believing Docker replaces ML coding
• Assuming Docker only speeds up training

2. Which of the following is the correct way to start a Docker container from an image named ml-image?

easy

A. docker run ml-image

B. docker start ml-image

C. docker build ml-image

D. docker create ml-image

Solution

1. Step 1: Identify the command to run a container

   The docker run command starts a container from an image.

2. Step 2: Understand other commands

   docker start starts stopped containers, docker build creates images, docker create creates containers but does not start them.

3. Final Answer:

   docker run ml-image -> Option A

4. Quick Check:

   Run container = docker run [OK]

Hint: Use 'docker run' to start containers from images [OK]

Common Mistakes:

• Using 'docker start' to run new containers
• Confusing 'docker build' with running containers
• Using 'docker create' without starting container

3. Given this Dockerfile snippet:

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . ./
CMD ["python", "train.py"]

What happens when you build and run this Docker image?

medium

A. The container installs dependencies and runs train.py automatically.

B. The container only copies files but does not install dependencies.

C. The container runs train.py without installing dependencies.

D. The container fails because requirements.txt is missing.

Solution

1. Step 1: Analyze Dockerfile steps

   The Dockerfile sets Python 3.12, copies requirements.txt, installs dependencies, copies code, then runs train.py.

2. Step 2: Understand build and run behavior

   Building installs dependencies; running executes train.py automatically as CMD defines the command.

3. Final Answer:

   The container installs dependencies and runs train.py automatically. -> Option A

4. Quick Check:

   Dockerfile CMD runs train.py after setup [OK]

Hint: CMD runs script after dependencies installed [OK]

Common Mistakes:

• Assuming dependencies are not installed
• Thinking CMD runs during build, not run
• Believing files are not copied before run

4. You wrote a Dockerfile but when running the container, your ML code fails with "ModuleNotFoundError". What is the most likely cause?

medium

A. You forgot to copy your code files into the image.

B. You did not expose the correct port in Dockerfile.

C. You used the wrong base image version.

D. You did not install the required Python packages.

Solution

1. Step 1: Understand ModuleNotFoundError meaning

   This error means Python cannot find a required package or module.

2. Step 2: Identify cause related to Dockerfile

   Not installing required packages (missing pip install) causes this error, not copying code or ports.

3. Final Answer:

   You did not install the required Python packages. -> Option D

4. Quick Check:

   ModuleNotFoundError = missing packages [OK]

Hint: Missing packages cause ModuleNotFoundError [OK]

Common Mistakes:

• Blaming missing code files instead of packages
• Confusing port exposure with module errors
• Assuming base image version always causes this

5. You want to ensure your ML training runs reproducibly with Docker, including specific Python version, dependencies, and data files. Which Dockerfile snippet best achieves this?

hard

A. FROM ubuntu:latest
RUN apt-get update
COPY train.py ./
CMD ["python", "train.py"]

B. FROM python:3.12
WORKDIR /app
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY data/ ./data/
COPY train.py ./
CMD ["python", "train.py"]

C. FROM python:latest
COPY train.py ./
CMD ["python", "train.py"]

D. FROM python:3.12
RUN pip install numpy
CMD ["python", "train.py"]

Solution

1. Step 1: Check for full environment setup

   FROM python:3.12 WORKDIR /app COPY requirements.txt ./ RUN pip install -r requirements.txt COPY data/ ./data/ COPY train.py ./ CMD ["python", "train.py"] sets Python 3.12, installs dependencies from requirements.txt, copies data and code, then runs training.

2. Step 2: Compare other options

   The other options miss dependencies, data files, or use generic Python versions, risking non-reproducibility.

3. Final Answer:

   FROM python:3.12 WORKDIR /app COPY requirements.txt ./ RUN pip install -r requirements.txt COPY data/ ./data/ COPY train.py ./ CMD ["python", "train.py"] -> Option B

4. Quick Check:

   Complete setup = reproducibility [OK]

Hint: Copy code, data, install deps, set Python version [OK]

Common Mistakes:

• Skipping dependency installation
• Not copying data files needed for training
• Using generic or latest Python versions