Docker for ML workloads in MLOps - Deep Dive

Overview - Docker for ML workloads

What is it?

Docker is a tool that packages software and its environment into a container. For machine learning (ML) workloads, Docker helps bundle the ML code, libraries, and dependencies so they run the same everywhere. This means you can train or deploy ML models without worrying about differences in computers or servers. It makes ML projects more reliable and easier to share.

Why it matters

Without Docker, ML projects often break when moved between computers because of missing or different software versions. This causes wasted time fixing environment issues instead of focusing on the ML itself. Docker solves this by creating a consistent, isolated space for ML workloads, making collaboration smoother and deployment faster. It helps teams deliver ML models to users reliably and repeatedly.

Where it fits

Before learning Docker for ML, you should understand basic ML workflows and how software dependencies work. After mastering Docker, you can explore Kubernetes for scaling ML workloads or CI/CD pipelines to automate ML model training and deployment.

Mental Model

Core Idea

Docker creates a portable, isolated box that holds your ML code and all its needs, so it runs the same anywhere.

Think of it like...

Imagine packing everything you need for a camping trip—tent, food, clothes—into a single backpack. No matter where you go, you have all essentials ready. Docker is like that backpack for ML projects.

┌─────────────────────────────┐
│        Host Machine         │
│ ┌─────────────────────────┐ │
│ │       Docker Engine      │ │
│ │ ┌─────────────────────┐ │ │
│ │ │   ML Container      │ │ │
│ │ │ ┌───────────────┐  │ │ │
│ │ │ │ ML Code &     │  │ │ │
│ │ │ │ Dependencies  │  │ │ │
│ │ │ └───────────────┘  │ │ │
│ │ └─────────────────────┘ │ │
│ └─────────────────────────┘ │
└─────────────────────────────┘

Build-Up - 7 Steps

1

Foundation

What is Docker and Containers

Concept: Introduce Docker as a tool that uses containers to package software and its environment.

Docker is software that creates containers. Containers are like small, lightweight boxes that hold your program and everything it needs to run. This means the program works the same on any computer with Docker installed. Unlike virtual machines, containers share the computer's system but keep the program isolated.

Result

Learners understand that Docker packages software and dependencies into containers for consistent execution.

Understanding containers as isolated, lightweight environments is key to grasping how Docker solves environment problems.

2

Foundation

Why ML Workloads Need Containers

Concept: Explain the challenges ML projects face with dependencies and environments.

ML projects use many libraries like TensorFlow or PyTorch, each with specific versions. Different computers might have different versions or missing libraries, causing errors. Containers bundle all these libraries with the ML code, so the project runs without errors anywhere.

Result

Learners see the problem of inconsistent ML environments and how containers prevent it.

Knowing the complexity of ML dependencies clarifies why containerization is essential for ML workloads.

3

Intermediate

Building a Docker Image for ML

🤔Before reading on: do you think a Docker image includes just your ML code or also the libraries and system tools? Commit to your answer.

Concept: Teach how to write a Dockerfile to create an image that includes ML code and dependencies.

A Dockerfile is a text file with instructions to build a Docker image. For ML, it starts from a base image like 'python:3.9', installs ML libraries, copies your code, and sets the command to run your ML script. Example: FROM python:3.9 RUN pip install tensorflow numpy COPY . /app WORKDIR /app CMD ["python", "train.py"]

Result

Learners can create Docker images that package ML code and dependencies.

Knowing how to build images lets you control exactly what your ML environment contains, ensuring consistency.

4

Intermediate

Running and Managing ML Containers

🤔Before reading on: do you think running a container changes your computer’s main system or keeps it separate? Commit to your answer.

Concept: Show how to run containers and manage their lifecycle without affecting the host system.

Use 'docker run' to start a container from your ML image. The container runs isolated from your main system, so it won’t change your computer’s files or settings. You can stop, restart, or remove containers easily. Example: docker run --rm my-ml-image The '--rm' flag removes the container after it stops.

Result

Learners can run ML workloads safely in containers and manage them.

Understanding container isolation prevents accidental changes to your main system and helps manage ML experiments cleanly.

5

Intermediate

Sharing ML Environments with Docker Hub

Concept: Explain how to share Docker images via public or private registries.

Docker Hub is a service to store and share Docker images. After building your ML image, you can upload it to Docker Hub. Others can download and run the exact same environment. Commands: docker tag my-ml-image username/my-ml-image:tag docker push username/my-ml-image:tag This makes collaboration easy and reproducible.

Result

Learners can share ML environments with teammates or deploy on servers.

Knowing how to share images enables teamwork and consistent deployment across different machines.

6

Advanced

Optimizing Docker Images for ML Workloads

🤔Before reading on: do you think smaller Docker images run faster or just save disk space? Commit to your answer.

Concept: Teach techniques to reduce image size and improve performance for ML containers.

Large images slow down transfer and startup. Use multi-stage builds to separate build tools from runtime. Choose slim base images like 'python:3.9-slim'. Cache dependencies to avoid reinstalling. Example snippet: FROM python:3.9-slim RUN pip install --no-cache-dir tensorflow numpy COPY . /app Smaller images start faster and use less storage.

Result

Learners create efficient ML Docker images that save time and resources.

Understanding image optimization improves ML workflow speed and resource use, critical for production.

7

Expert

Handling GPU Support in ML Containers

🤔Before reading on: do you think Docker containers can use your computer’s GPU by default? Commit to your answer.

Concept: Explain how to enable GPU access inside Docker containers for ML training acceleration.

By default, containers cannot use GPUs. You need NVIDIA’s container toolkit and drivers installed on the host. Run containers with special flags: docker run --gpus all my-ml-image This allows ML code inside the container to use GPUs for faster training. The container must have GPU libraries like CUDA installed.

Result

Learners can run GPU-accelerated ML workloads inside Docker containers.

Knowing how to enable GPU access bridges container isolation with hardware acceleration, essential for real ML workloads.

Under the Hood

Docker uses OS-level virtualization to create containers. It shares the host OS kernel but isolates processes, file systems, and network using namespaces and control groups (cgroups). This isolation ensures containers run independently without interfering with each other or the host. Docker images are layered filesystems built from instructions in Dockerfiles. When running a container, Docker combines these layers into a single view and starts the process inside the isolated environment.

Why designed this way?

Docker was designed to be lightweight and fast compared to full virtual machines. Using OS-level features avoids the overhead of running separate OS instances. Layered images allow reuse of common parts, saving space and speeding up builds. This design balances isolation with performance, making it ideal for packaging complex ML environments that need consistency without heavy resource use.

Host OS Kernel
┌─────────────────────────────┐
│ Docker Engine               │
│ ┌───────────────┐           │
│ │ Namespaces    │           │
│ │ & cgroups     │           │
│ └───────────────┘           │
│ ┌───────────────┐           │
│ │ Container 1   │           │
│ │ (ML workload) │           │
│ └───────────────┘           │
│ ┌───────────────┐           │
│ │ Container 2   │           │
│ │ (Other app)   │           │
│ └───────────────┘           │
└─────────────────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Do you think Docker containers are full virtual machines? Commit yes or no.

Common Belief: Docker containers are just like virtual machines with their own full operating system.

Reality: Docker containers share the host OS kernel and isolate only processes and filesystems, making them much lighter than virtual machines.

Why it matters: Thinking containers are heavy like VMs leads to overestimating resource needs and misunderstanding container startup speed.

Quick: Can Docker containers automatically use your GPU without extra setup? Commit yes or no.

Common Belief: Docker containers can use GPUs by default just like the host system.

Reality: Containers need special drivers and runtime setup to access GPUs; otherwise, they cannot use GPU hardware.

Why it matters: Assuming GPU works by default causes wasted time debugging ML training performance issues.

Quick: Does sharing a Docker image guarantee your ML code will run identically everywhere? Commit yes or no.

Common Belief: If I share a Docker image, my ML code will always run exactly the same on any machine.

Reality: While Docker ensures environment consistency, differences in hardware (like GPUs) or external data sources can still cause variations.

Why it matters: Overreliance on Docker images alone can lead to overlooked issues in production related to hardware or data.

Quick: Is a smaller Docker image always faster to run? Commit yes or no.

Common Belief: Smaller Docker images always start and run faster than larger ones.

Reality: Smaller images reduce download and startup time, but runtime speed depends on the ML code and hardware, not image size alone.

Why it matters: Focusing only on image size can distract from optimizing actual ML workload performance.

Expert Zone

1

Docker image layers cache can cause stale dependencies if not managed carefully, leading to hard-to-debug ML bugs.

2

Using multi-stage builds not only reduces image size but also improves security by excluding build tools from the final image.

3

GPU support requires matching host driver versions with container CUDA libraries; mismatches cause silent failures or crashes.

When NOT to use

Docker is not ideal when ultra-low latency or direct hardware access is required beyond GPU support. In such cases, bare-metal deployment or specialized orchestration like Kubernetes with device plugins is better. Also, for very simple ML scripts without complex dependencies, virtual environments might suffice.

Production Patterns

In production, ML teams use Docker images combined with CI/CD pipelines to automate training and deployment. Images are versioned and stored in private registries. GPU-enabled containers run on cloud or on-prem clusters managed by Kubernetes. Monitoring and logging are integrated to track ML model performance and resource use.

Connections

Virtual Machines

Docker containers are a lightweight alternative to virtual machines.

Understanding the difference helps choose the right isolation tool for ML workloads balancing performance and security.

Continuous Integration/Continuous Deployment (CI/CD)

Docker images are often built and tested automatically in CI/CD pipelines for ML projects.

Knowing Docker enables smoother automation of ML model updates and deployment.

Supply Chain Packaging

Docker containers bundle all parts needed to run ML code, similar to how supply chains package and deliver goods reliably.

Recognizing this connection highlights the importance of packaging completeness and consistency in both software and physical goods delivery.

Common Pitfalls

#1 Not specifying exact library versions in Dockerfile causes inconsistent ML environments.

Wrong approach:

RUN pip install tensorflow numpy

Correct approach:

RUN pip install tensorflow==2.12.0 numpy==1.24.3

Root cause:

Assuming latest versions are always compatible leads to unexpected breaks when dependencies update.

#2 Running containers without cleaning up leads to many stopped containers consuming disk space.

Wrong approach:

docker run my-ml-image

Correct approach:

docker run --rm my-ml-image

Root cause:

Not using '--rm' flag or manual cleanup causes clutter and wasted storage.

#3 Trying to use GPU inside container without installing NVIDIA container toolkit causes errors.

Wrong approach:

docker run --gpus all my-ml-image (without toolkit installed)

Correct approach:

Install NVIDIA container toolkit on host, then run: docker run --gpus all my-ml-image

Root cause:

Assuming GPU access works out-of-the-box ignores necessary host setup.

Key Takeaways

Docker containers package ML code and all dependencies into a portable, consistent environment.

Containers share the host OS kernel but isolate processes and files, making them lightweight compared to virtual machines.

Building Docker images with exact dependency versions ensures reproducible ML environments.

GPU support in Docker requires special setup beyond just running containers.

Optimizing Docker images and managing containers properly improves ML workflow efficiency and reliability.

Practice

1. What is the main benefit of using Docker for ML workloads?

easy

A. It provides a graphical interface for ML model training.

B. It automatically improves the accuracy of ML models.

C. It replaces the need for data preprocessing.

D. It packages the ML project with all dependencies to run anywhere.

Solution

1. Step 1: Understand Docker's role in ML

   Docker packages the ML project with all needed tools and code, ensuring consistency.

2. Step 2: Identify the main benefit

   This packaging allows the ML workload to run the same way on any machine without setup issues.

3. Final Answer:

   It packages the ML project with all dependencies to run anywhere. -> Option D

4. Quick Check:

   Docker ensures consistent ML environment = D [OK]

Hint: Docker bundles code and tools for consistent runs anywhere [OK]

Common Mistakes:

• Thinking Docker improves model accuracy
• Believing Docker replaces data preprocessing
• Assuming Docker provides a GUI for training

2. Which of the following is the correct syntax to start a Docker container named ml_container from an image called ml_image?

easy

A. docker start ml_image --name ml_container

B. docker create ml_image ml_container

C. docker run --name ml_container ml_image

D. docker build ml_container ml_image

Solution

1. Step 1: Recall Docker run command syntax

   The command to start a container with a name is: docker run --name [container_name] [image_name].

2. Step 2: Match the correct syntax

   docker run --name ml_container ml_image matches this syntax exactly, starting a container named ml_container from ml_image.

3. Final Answer:

   docker run --name ml_container ml_image -> Option C

4. Quick Check:

   docker run --name container image = B [OK]

Hint: Use 'docker run --name' to start named containers [OK]

Common Mistakes:

• Using docker start instead of docker run to create container
• Confusing docker build with running containers
• Wrong order of arguments in command

3. Given this Dockerfile snippet for an ML project:

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . ./
CMD ["python", "train.py"]

What happens when you run docker build -t ml_train . followed by docker run ml_train?

medium

A. The container only copies files but does not run train.py.

B. The container installs dependencies and runs train.py automatically.

C. The build command fails due to missing CMD syntax.

D. The container runs but does not install dependencies.

Solution

1. Step 1: Analyze Dockerfile commands

   The Dockerfile installs Python 3.12, sets /app as working directory, copies requirements.txt, installs dependencies, copies all files, then sets command to run train.py.

2. Step 2: Understand build and run behavior

   docker build creates an image with dependencies installed. docker run starts a container that runs train.py automatically as CMD is set.

3. Final Answer:

   The container installs dependencies and runs train.py automatically. -> Option B

4. Quick Check:

   Dockerfile CMD runs train.py after build and run = A [OK]

Hint: CMD runs train.py after build and run commands [OK]

Common Mistakes:

• Thinking CMD is ignored during run
• Assuming build fails without explicit entrypoint
• Believing dependencies install at run time

4. You wrote this Dockerfile for your ML project:

FROM python:3.12
COPY . /app
WORKDIR /app
RUN pip install -r requirements.txt
CMD python train.py

When building the image, you get an error: pip: command not found. What is the likely cause?

medium

A. The base image python:3.12 does not include pip by default.

B. The COPY command is incorrect and did not copy requirements.txt.

C. The CMD syntax is wrong and causes build failure.

D. The WORKDIR is set after COPY, causing path issues.

Solution

1. Step 1: Check base image contents

   Some python base images do not include pip by default, causing 'pip: command not found' error.

2. Step 2: Verify other commands

   COPY and WORKDIR are correct; CMD syntax is valid for shell form. The error points to missing pip in base image.

3. Final Answer:

   The base image python:3.12 does not include pip by default. -> Option A

4. Quick Check:

   Missing pip in base image causes error = A [OK]

Hint: Check if base image includes pip before installing packages [OK]

Common Mistakes:

• Blaming COPY command for pip error
• Thinking CMD syntax causes build error
• Ignoring base image contents

5. You want to optimize your Dockerfile for faster ML model training iterations by caching dependencies. Which change helps achieve this?

hard

A. Copy only requirements.txt and run pip install before copying the rest of the code.

B. Copy all files first, then run pip install to include all dependencies.

C. Run pip install after CMD to delay installation.

D. Use docker run to install dependencies each time the container starts.

Solution

1. Step 1: Understand Docker layer caching

   Docker caches layers. If requirements.txt changes, only pip install layer rebuilds, speeding up builds.

2. Step 2: Apply caching best practice

   Copying requirements.txt and installing dependencies before copying other code avoids reinstalling packages when code changes.

3. Final Answer:

   Copy only requirements.txt and run pip install before copying the rest of the code. -> Option A

4. Quick Check:

   Separate requirements.txt copy for caching = C [OK]

Hint: Copy requirements.txt first to cache pip install layer [OK]

Common Mistakes:

• Copying all files before pip install causing cache misses
• Running pip install after CMD which never executes during build
• Installing dependencies at container start wasting time