Why the training loop is explicit in PyTorch - Challenge Your Understanding

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why does PyTorch require an explicit training loop?

PyTorch users write their own training loops instead of using a built-in one. Why is this explicit training loop important?

A It allows users to customize every step of training, like data loading, forward pass, loss calculation, and backpropagation.

B Because PyTorch only works with pre-trained models and does not support training from scratch.

C To force users to write longer code and make training slower for better learning.

D Because PyTorch does not support automatic differentiation, so users must manually compute gradients.

💡 Hint

Think about flexibility and control during model training.

PyTorch's explicit training loop gives users full control over each step, enabling easy customization and debugging. It supports automatic differentiation, so gradients are computed automatically, but users decide how to use them.

❓ Predict Output

intermediate

Output of a simple PyTorch training loop snippet

What will be the printed output after running this PyTorch training loop snippet?

PyTorch

import torch
import torch.nn as nn

model = nn.Linear(2, 1)
optimizer = torch.optim.SGD(model.parameters(), lr=0.1)
criterion = nn.MSELoss()

inputs = torch.tensor([[1.0, 2.0]])
target = torch.tensor([[1.0]])

optimizer.zero_grad()
output = model(inputs)
loss = criterion(output, target)
loss.backward()
optimizer.step()

print(round(loss.item(), 4))

A 0.0

B 0.25

C 0.5

D 1.0

💡 Hint

Think about the initial random weights and the MSE loss formula.

The initial weights are random, so the output is not equal to the target. The MSE loss is roughly 0.5 for this input and target after one forward pass.

❓ Model Choice

advanced

Choosing the right model update step in PyTorch training loop

In a PyTorch training loop, which step correctly updates the model parameters after computing gradients?

A model.backward()

B optimizer.step()

C loss.backward()

D optimizer.zero_grad()

💡 Hint

Which function applies the computed gradients to change model weights?

optimizer.step() applies the gradients to update model parameters. loss.backward() computes gradients, optimizer.zero_grad() clears old gradients, and model.backward() is not a valid method.

❓ Hyperparameter

advanced

Effect of learning rate in explicit PyTorch training loop

What happens if the learning rate in the optimizer is set too high in a PyTorch training loop?

A The model parameters may update too much, causing training to diverge or become unstable.

B The model will train faster and always reach the best accuracy immediately.

C The gradients will not be computed, so the model won't learn anything.

D The loss function will automatically adjust to compensate for the high learning rate.

💡 Hint

Think about how big steps affect learning stability.

A very high learning rate can cause the model to overshoot the best parameters, making training unstable or diverging instead of converging.

🔧 Debug

expert

Identifying the error in this PyTorch training loop snippet

What error will this PyTorch training loop snippet raise?

PyTorch

import torch
import torch.nn as nn

model = nn.Linear(3, 1)
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)
criterion = nn.MSELoss()

inputs = torch.tensor([[1.0, 2.0]])  # Only 2 features instead of 3
target = torch.tensor([[1.0]])

optimizer.zero_grad()
output = model(inputs)
loss = criterion(output, target)
loss.backward()
optimizer.step()

A ValueError: target and output shapes do not match

B TypeError: optimizer.step() missing required positional argument

C No error, runs successfully

D RuntimeError: size mismatch in linear layer input

💡 Hint

Check the input size vs model expected input size.

The model expects input with 3 features, but the input tensor has only 2 features, causing a size mismatch error during the forward pass.