You are planning GPU resources for training a deep learning model. The model requires 12GB of GPU memory per training batch. You want to train with a batch size of 64. How much total GPU memory is needed if you want to run the training on a single GPU without memory overflow?
Multiply the memory per batch by the batch size to get total memory needed.
The memory requirement scales with batch size: 12GB × 64 = 768GB.
You want to speed up training by using multiple GPUs in parallel. Which GPU setup is best for minimizing communication overhead between GPUs?
Consider the speed and latency of connections between GPUs.
GPUs connected via PCIe on the same motherboard have the fastest communication, reducing overhead.
You have a GPU with 24GB memory. Your model uses 8GB per batch of size 32. You want to increase batch size but cannot exceed GPU memory. What is the maximum batch size you can use?
Calculate memory per sample from 8GB per 32 samples, then max batch size = 24GB / mem_per_sample.
8GB for batch size 32 means 8/32 = 0.25GB per sample. Max batch size = 24GB / 0.25GB = 96.
You monitor GPU utilization during training and see it averages 30%. What does this indicate about your GPU usage?
Consider what low GPU utilization means for training speed.
30% utilization means GPU is idle much of the time, so training is slower than it could be.
You set up training on 4 GPUs but notice training is slower than on a single GPU. Which is the most likely cause?
Think about what slows down multi-GPU training besides memory or utilization.
Communication overhead can cause slowdowns if GPUs spend too much time syncing data.