Installation and GPU setup in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding GPU Availability in PyTorch

Which PyTorch command correctly checks if a GPU is available for use?

A torch.cuda.is_available()

B torch.gpu.is_ready()

C torch.device('gpu').is_available()

D torch.cuda.check_gpu()

💡 Hint

Look for the official PyTorch method that returns a boolean indicating GPU availability.

The correct method to check GPU availability in PyTorch is torch.cuda.is_available(). Other options are not valid PyTorch commands.

❓ Predict Output

intermediate

Output of Device Assignment Code

What is the output of this PyTorch code snippet?

import torch
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
tensor = torch.tensor([1, 2, 3]).to(device)
print(tensor.device)

A cpu

B cuda:0

C cuda

D cuda:1

💡 Hint

The tensor is moved to the first GPU if available, otherwise CPU.

If a GPU is available, the tensor's device will be cuda:0. Otherwise, it would be cpu. The code prints the device property which includes the device type and index.

❓ Model Choice

advanced

Choosing the Correct Device for Model Training

You want to train a deep learning model using PyTorch on a machine with multiple GPUs. Which device assignment is best to ensure the model uses the first GPU?

A device = torch.device('cuda')

B device = torch.device('cpu')

C device = torch.device('cuda:1')

D device = torch.device('cuda:0')

💡 Hint

Remember GPU indices start at 0. To use the first GPU explicitly, specify its index.

Using cuda:0 explicitly selects the first GPU. cuda defaults to cuda:0 but being explicit is better for clarity. cuda:1 selects the second GPU. cpu uses the processor only.

❓ Hyperparameter

advanced

Effect of Batch Size on GPU Memory Usage

When training a model on GPU, increasing the batch size will most likely:

A Cause the GPU to switch to CPU automatically

B Have no effect on GPU memory usage

C Increase GPU memory usage

D Decrease GPU memory usage

💡 Hint

Think about how many data samples are processed at once and how that affects memory.

Increasing batch size means more data is processed simultaneously, which requires more GPU memory to store inputs, activations, and gradients.

🔧 Debug

expert

Identifying the Cause of CUDA Out of Memory Error

Given this PyTorch training loop snippet, what is the most likely cause of a CUDA Out of Memory error?

for data, target in dataloader:
    optimizer.zero_grad()
    output = model(data.to(device))
    loss = loss_fn(output, target.to(device))
    loss.backward()
    optimizer.step()

A Accumulating gradients without clearing them each iteration

B Not moving data and target tensors to the GPU device

C Not calling <code>optimizer.zero_grad()</code> before backward pass

D Using <code>loss.backward()</code> instead of <code>loss.forward()</code>

💡 Hint

Consider what happens if gradients keep adding up every iteration without reset.

If gradients are not cleared each iteration, they accumulate and consume more GPU memory, eventually causing an out of memory error. The code calls optimizer.zero_grad() correctly, so the error is likely from missing or misplaced gradient clearing in other code versions.

Practice

1. What is the correct command to install PyTorch using pip?

easy

A. pip install torch-gpu-support

B. pip install pytorch-gpu

C. pip install torch-cuda

D. pip install torch torchvision torchaudio

Solution

1. Step 1: Understand PyTorch installation command

   The official PyTorch installation command is pip install torch torchvision torchaudio.

2. Step 2: Check other options

   Other options like pytorch-gpu or torch-cuda are not valid package names for PyTorch installation.

3. Final Answer:

   pip install torch torchvision torchaudio -> Option D

4. Quick Check:

   Official PyTorch install command = pip install torch torchvision torchaudio [OK]

Hint: Use official PyTorch install command with torch and torchvision [OK]

Common Mistakes:

• Trying to install non-existing packages like pytorch-gpu
• Forgetting to include torchvision or torchaudio
• Using conda command instead of pip

2. Which of the following Python code snippets correctly checks if a GPU is available in PyTorch?

easy

A. torch.gpu.available()

B. torch.cuda.is_available()

C. torch.cuda.check_gpu()

D. torch.device('gpu').is_available()

Solution

1. Step 1: Recall PyTorch GPU availability function

   PyTorch provides torch.cuda.is_available() to check if a GPU is accessible.

2. Step 2: Verify other options

   Functions like torch.gpu.available(), torch.cuda.check_gpu(), and torch.device('gpu').is_available() do not exist in PyTorch.

3. Final Answer:

   torch.cuda.is_available() -> Option B

4. Quick Check:

   GPU check function = torch.cuda.is_available() [OK]

Hint: Use torch.cuda.is_available() to check GPU presence [OK]

Common Mistakes:

• Using incorrect function names like torch.gpu.available()
• Trying to check GPU with torch.device('gpu')
• Confusing GPU check with device creation

3. What will be the output of the following code if a GPU is available?

import torch
print(torch.cuda.is_available())

medium

A. True

B. False

C. None

D. Error

Solution

1. Step 1: Understand torch.cuda.is_available() output

   This function returns a boolean: True if a GPU is available, otherwise False.

2. Step 2: Given GPU is available

   The code prints the result of torch.cuda.is_available(), so it will print True.

3. Final Answer:

   True -> Option A

4. Quick Check:

   GPU available? = True [OK]

Hint: If GPU exists, torch.cuda.is_available() returns True [OK]

Common Mistakes:

• Expecting string output instead of boolean
• Confusing True/False values
• Assuming function raises error if no GPU

4. Identify the error in this code snippet for checking GPU availability:

import torch
if torch.cuda.is_available:
    print("GPU is ready")
else:
    print("GPU not found")

medium

A. Missing parentheses after is_available

B. Incorrect import statement

C. Wrong print syntax

D. No error, code is correct

Solution

1. Step 1: Check function call syntax

   torch.cuda.is_available is a function and must be called with parentheses: torch.cuda.is_available().

2. Step 2: Verify other parts of code

   Import and print statements are correct. The only error is missing parentheses.

3. Final Answer:

   Missing parentheses after is_available -> Option A

4. Quick Check:

   Function call needs () = Missing parentheses [OK]

Hint: Remember to add () when calling functions like is_available [OK]

Common Mistakes:

• Forgetting parentheses on function calls
• Assuming is_available is a property, not a method
• Misreading indentation or print syntax

5. You want to run your PyTorch model on GPU if available, otherwise on CPU. Which code snippet correctly sets the device?

hard

A. device = torch.device('cuda:0' if torch.cuda.is_available else 'cpu:0')

B. device = torch.device('gpu' if torch.cuda.is_available() else 'cpu')

C. device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

D. device = torch.device('cuda:1' if torch.cuda.is_available() else 'cpu')

Solution

1. Step 1: Understand device selection syntax

   PyTorch uses torch.device('cuda') for GPU and torch.device('cpu') for CPU.

2. Step 2: Check conditional device assignment

   The correct way is to use 'cuda' if GPU is available, else 'cpu'. device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') uses this correctly.

3. Step 3: Analyze other options

   device = torch.device('cuda:0' if torch.cuda.is_available else 'cpu:0') omits parentheses after torch.cuda.is_available, so the function is not called (function object is truthy), always selecting GPU even if unavailable at runtime. device = torch.device('gpu' if torch.cuda.is_available() else 'cpu') uses invalid 'gpu' string. device = torch.device('cuda:1' if torch.cuda.is_available() else 'cpu') forces 'cuda:1' which may not exist.

4. Final Answer:

   device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') -> Option C

5. Quick Check:

   Use 'cuda' or 'cpu' strings correctly for device [OK]

Hint: Use 'cuda' string, not 'gpu', for device selection [OK]

Common Mistakes:

• Forgetting parentheses after torch.cuda.is_available()
• Using 'gpu' instead of 'cuda' in device string
• Hardcoding GPU index without checking availability