GPU support in containers in MLOps - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the main purpose of GPU support in containers?

GPU support in containers allows applications running inside containers to use the computer's GPU hardware for faster processing, especially useful for tasks like machine learning and data processing.

beginner

Which NVIDIA tool helps containers access GPU resources easily?

NVIDIA provides the 'NVIDIA Container Toolkit' which enables containers to use NVIDIA GPUs by integrating GPU drivers and libraries inside the container environment.

intermediate

How do you run a Docker container with GPU support using NVIDIA runtime?

You run the container with the flag: docker run --gpus all <image>. This tells Docker to give the container access to all GPUs on the host machine.

beginner

Why can't containers use GPUs by default without special setup?

Containers are isolated and do not have direct access to hardware like GPUs. Special drivers and runtimes are needed to bridge this gap and allow GPU usage inside containers.

intermediate

Name one common environment variable used inside containers to detect GPU availability.

The environment variable CUDA_VISIBLE_DEVICES is often used to specify which GPUs a containerized application can see and use.

Which command flag enables GPU support when running a Docker container?

A --use-gpu

B --enable-gpu

C --gpus all

D --gpu-access

The correct flag to enable GPU support in Docker is '--gpus all'.

What does the NVIDIA Container Toolkit provide?

A A way for containers to access NVIDIA GPUs

B A new GPU hardware

C A container orchestration tool

D A GPU monitoring dashboard

The NVIDIA Container Toolkit allows containers to use NVIDIA GPUs by providing necessary drivers and libraries.

Why do containers need special setup to use GPUs?

A Because containers only run on CPUs

B Because containers isolate hardware access by default

C Because GPUs are not compatible with containers

D Because GPUs require internet access

Containers isolate hardware, so special drivers and runtimes are needed to allow GPU access.

Which environment variable controls GPU visibility inside a container?

A GPU_ACCESS_LEVEL

B NVIDIA_GPU_FLAG

C CONTAINER_GPU

D CUDA_VISIBLE_DEVICES

CUDA_VISIBLE_DEVICES specifies which GPUs are visible to the containerized application.

What is a common use case for GPU support in containers?

A Machine learning model training

B Simple text editing

C Web page hosting

D File storage

GPUs speed up heavy computations like machine learning training, making this a common use case.

Explain how GPU support works in containers and why it is important.

Think about how containers normally isolate hardware and what tools help bridge that gap.

Describe the steps to run a container with GPU support using Docker.

Consider what setup is needed on the host and the command to start the container.