Installation and GPU setup in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding GPU Availability in PyTorch

Which PyTorch command correctly checks if a GPU is available for use?

A torch.cuda.is_available()

B torch.gpu.is_ready()

C torch.device('gpu').is_available()

D torch.cuda.check_gpu()

💡 Hint

Look for the official PyTorch method that returns a boolean indicating GPU availability.

The correct method to check GPU availability in PyTorch is torch.cuda.is_available(). Other options are not valid PyTorch commands.

❓ Predict Output

intermediate

Output of Device Assignment Code

What is the output of this PyTorch code snippet?

import torch
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
tensor = torch.tensor([1, 2, 3]).to(device)
print(tensor.device)

A cpu

B cuda:0

C cuda

D cuda:1

💡 Hint

The tensor is moved to the first GPU if available, otherwise CPU.

If a GPU is available, the tensor's device will be cuda:0. Otherwise, it would be cpu. The code prints the device property which includes the device type and index.

❓ Model Choice

advanced

Choosing the Correct Device for Model Training

You want to train a deep learning model using PyTorch on a machine with multiple GPUs. Which device assignment is best to ensure the model uses the first GPU?

A device = torch.device('cuda')

B device = torch.device('cpu')

C device = torch.device('cuda:1')

D device = torch.device('cuda:0')

💡 Hint

Remember GPU indices start at 0. To use the first GPU explicitly, specify its index.

Using cuda:0 explicitly selects the first GPU. cuda defaults to cuda:0 but being explicit is better for clarity. cuda:1 selects the second GPU. cpu uses the processor only.

❓ Hyperparameter

advanced

Effect of Batch Size on GPU Memory Usage

When training a model on GPU, increasing the batch size will most likely:

A Cause the GPU to switch to CPU automatically

B Have no effect on GPU memory usage

C Increase GPU memory usage

D Decrease GPU memory usage

💡 Hint

Think about how many data samples are processed at once and how that affects memory.

Increasing batch size means more data is processed simultaneously, which requires more GPU memory to store inputs, activations, and gradients.

🔧 Debug

expert

Identifying the Cause of CUDA Out of Memory Error

Given this PyTorch training loop snippet, what is the most likely cause of a CUDA Out of Memory error?

for data, target in dataloader:
    optimizer.zero_grad()
    output = model(data.to(device))
    loss = loss_fn(output, target.to(device))
    loss.backward()
    optimizer.step()

A Accumulating gradients without clearing them each iteration

B Not moving data and target tensors to the GPU device

C Not calling <code>optimizer.zero_grad()</code> before backward pass

D Using <code>loss.backward()</code> instead of <code>loss.forward()</code>

💡 Hint

Consider what happens if gradients keep adding up every iteration without reset.

If gradients are not cleared each iteration, they accumulate and consume more GPU memory, eventually causing an out of memory error. The code calls optimizer.zero_grad() correctly, so the error is likely from missing or misplaced gradient clearing in other code versions.