Forward pass, loss, backward, step in PyTorch - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to perform a forward pass using the model.

PyTorch

outputs = model([1])

Drag options to blanks, or click blank then click option'

Aloss

Blabels

Coptimizer

Dinputs

Attempts:

3 left

💡 Hint

Common Mistakes

Using labels instead of inputs for the forward pass.

Passing the optimizer to the model.

The forward pass uses the input data to get predictions from the model.

2fill in blank

medium

Complete the code to calculate the loss between outputs and labels.

PyTorch

loss = criterion(outputs, [1])

Drag options to blanks, or click blank then click option'

Alabels

Binputs

Coptimizer

Dmodel

Attempts:

3 left

💡 Hint

Common Mistakes

Using inputs instead of labels for loss calculation.

Passing the model or optimizer instead of labels.

Loss compares the model's outputs to the true labels to measure error.

3fill in blank

hard

Fix the error in the code to perform backpropagation.

PyTorch

loss.[1]()

Drag options to blanks, or click blank then click option'

Astep

Bbackward

Cforward

Dzero_grad

Attempts:

3 left

💡 Hint

Common Mistakes

Calling step() on loss instead of optimizer.

Calling zero_grad() on loss.

Calling backward() computes gradients for backpropagation.

4fill in blank

hard

Fill both blanks to reset gradients and update model parameters.

PyTorch

optimizer.[1]()
optimizer.[2]()

Drag options to blanks, or click blank then click option'

Azero_grad

Bstep

Cbackward

Dforward

Attempts:

3 left

💡 Hint

Common Mistakes

Calling backward() or forward() on optimizer.

Not clearing gradients before the step.

First, gradients are cleared with zero_grad(). Then, step() updates the model weights.

5fill in blank

hard

Fill all three blanks to complete a training step: forward pass, loss calculation, and backward pass.

PyTorch

outputs = model([1])
loss = criterion(outputs, [2])
loss.[3]()

Drag options to blanks, or click blank then click option'

Ainputs

Blabels

Cbackward

Doptimizer

Attempts:

3 left

💡 Hint

Common Mistakes

Mixing up inputs and labels in loss calculation.

Forgetting to call backward on loss.

The training step starts with inputs to get outputs, then calculates loss with labels, and finally calls backward() to compute gradients.

Practice

1. What is the main purpose of the forward pass in a PyTorch model?

easy

A. To compute gradients for backpropagation

B. To update the model weights after training

C. To run input data through the model and get predictions

D. To calculate the loss between predictions and targets

Solution

1. Step 1: Understand the forward pass role

   The forward pass takes input data and processes it through the model layers to produce predictions.

2. Step 2: Differentiate from other steps

   Updating weights is done by the optimizer step, loss calculation measures error, and backward computes gradients.

3. Final Answer:

   To run input data through the model and get predictions -> Option C

4. Quick Check:

   Forward pass = predictions [OK]

Hint: Forward pass means getting predictions from inputs [OK]

Common Mistakes:

• Confusing forward pass with weight update
• Mixing loss calculation with forward pass
• Thinking backward computes predictions

2. Which of the following is the correct PyTorch code to perform a backward pass on a loss tensor named loss?

easy

A. loss.step()

B. loss.backward()

C. loss.forward()

D. loss.update()

Solution

1. Step 1: Identify the backward method

   In PyTorch, calling loss.backward() computes gradients for all tensors with requires_grad=True.

2. Step 2: Check other options

   step() is for optimizers, forward() is for models, and update() is not a PyTorch method.

3. Final Answer:

   loss.backward() -> Option B

4. Quick Check:

   Backward pass uses loss.backward() [OK]

Hint: Use loss.backward() to compute gradients [OK]

Common Mistakes:

• Using step() on loss instead of optimizer
• Calling forward() on loss tensor
• Trying to update loss directly

3. Given the following PyTorch code snippet, what will be printed?

import torch
import torch.nn as nn

model = nn.Linear(2, 1)
optimizer = torch.optim.SGD(model.parameters(), lr=0.1)

inputs = torch.tensor([[1.0, 2.0]])
targets = torch.tensor([[1.0]])

outputs = model(inputs)
loss_fn = nn.MSELoss()
loss = loss_fn(outputs, targets)

loss.backward()
optimizer.step()

print(loss.item())

medium

A. A positive float value representing the loss before weight update

B. Zero, because optimizer.step() resets the loss

C. A tensor object, not a float

D. An error because backward() must be called after step()

Solution

1. Step 1: Understand loss calculation and printing

   The loss is computed before backward and step, so loss.item() returns the scalar loss value as a float.

2. Step 2: Confirm optimizer.step() effect

   Calling optimizer.step() updates weights but does not change the already computed loss tensor value.

3. Final Answer:

   A positive float value representing the loss before weight update -> Option A

4. Quick Check:

   loss.item() shows loss before update [OK]

Hint: Loss printed is before optimizer updates weights [OK]

Common Mistakes:

• Thinking loss becomes zero after optimizer.step()
• Expecting loss to be a tensor when printed
• Calling backward() after step() causes error

4. Identify the error in this PyTorch training snippet:

optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
outputs = model(inputs)
loss = loss_fn(outputs, targets)
optimizer.step()
loss.backward()

medium

A. Missing optimizer.zero_grad() before backward

B. Model parameters not passed to optimizer

C. Using wrong loss function

D. Calling optimizer.step() before loss.backward()

Solution

1. Step 1: Check order of backward and step

   Gradients must be computed by loss.backward() before calling optimizer.step() to update weights.

2. Step 2: Analyze given code order

   The code calls optimizer.step() before loss.backward(), so weights update happens without gradients.

3. Final Answer:

   Calling optimizer.step() before loss.backward() -> Option D

4. Quick Check:

   Backward before step is required [OK]

Hint: Always call loss.backward() before optimizer.step() [OK]

Common Mistakes:

• Swapping backward and step order
• Forgetting to zero gradients
• Assuming step() computes gradients

5. You want to train a PyTorch model for one step with input x and target y. Which sequence of commands correctly performs forward pass, loss calculation, backward pass, and weight update?

hard

A. optimizer.zero_grad(); outputs = model(x); loss = loss_fn(outputs, y); loss.backward(); optimizer.step()

B. outputs = model(x); loss = loss_fn(outputs, y); optimizer.step(); loss.backward(); optimizer.zero_grad()

C. loss = loss_fn(x, y); outputs = model(x); optimizer.zero_grad(); loss.backward(); optimizer.step()

D. optimizer.step(); optimizer.zero_grad(); outputs = model(x); loss = loss_fn(outputs, y); loss.backward()

Solution

1. Step 1: Understand correct training step order

   The correct order is: clear gradients, forward pass, compute loss, backward pass, then update weights.

2. Step 2: Check each option's sequence

   optimizer.zero_grad(); outputs = model(x); loss = loss_fn(outputs, y); loss.backward(); optimizer.step() follows the correct order. Others call optimizer.step() too early or compute loss before forward pass.

3. Final Answer:

   optimizer.zero_grad(); outputs = model(x); loss = loss_fn(outputs, y); loss.backward(); optimizer.step() -> Option A

4. Quick Check:

   Zero grad -> forward -> loss -> backward -> step [OK]

Hint: Clear grads, forward, loss, backward, then step [OK]

Common Mistakes:

• Calling optimizer.step() before backward
• Not zeroing gradients before backward
• Computing loss before forward pass