Backward pass (loss.backward) in PyTorch - Model Metrics & Evaluation

Metrics & Evaluation - Backward pass (loss.backward)

Which metric matters for Backward pass (loss.backward) and WHY

The backward pass itself does not produce a metric like accuracy or precision. Instead, it calculates gradients that help the model learn by updating weights to reduce the loss. The key metric to watch is the loss value before calling loss.backward(). A decreasing loss means the backward pass is helping the model improve.

Confusion matrix or equivalent visualization

The backward pass is part of training, not evaluation, so it does not have a confusion matrix. Instead, you can track the loss curve over training steps to see if the model is learning:

Epoch | Loss
------+-------
  1   | 0.85
  2   | 0.65
  3   | 0.50
  4   | 0.40
  5   | 0.35
    

A steady drop in loss means gradients from loss.backward() are helping the model improve.

Precision vs Recall tradeoff with concrete examples

The backward pass affects all metrics indirectly by improving the model. For example, if the model is a spam filter, a good backward pass helps reduce both false positives and false negatives by lowering loss. This improves both precision (correct spam detection) and recall (catching all spam).

Think of the backward pass as the engine that powers learning. If it works well, the model balances precision and recall better over time.

What "good" vs "bad" metric values look like for this use case

Since loss.backward() computes gradients, a "good" backward pass means:

• Loss decreases steadily over epochs.
• Gradients are not zero or exploding (very large).
• Model weights update smoothly.

A "bad" backward pass might show:

• Loss stays the same or increases.
• Gradients vanish (zero) or explode (huge values).
• Training does not improve metrics like accuracy.

Metrics pitfalls

• Zero gradients: Forgetting to call optimizer.zero_grad() before loss.backward() causes gradients to accumulate incorrectly.
• Data leakage: Using test data during training can make loss look better but hurts real performance.
• Overfitting: Loss on training data drops but validation loss rises, meaning the model memorizes instead of learning.
• Incorrect loss function: Using a loss that does not match the task can mislead training.

Self-check

Your model has 98% accuracy but 12% recall on fraud cases. Is it good for production?

Answer: No. The backward pass may reduce loss, but low recall means the model misses most fraud cases. This is dangerous because catching fraud is critical. You need to improve training (backward pass and loss) to increase recall, even if accuracy drops.

Key Result

The backward pass enables learning by computing gradients; its success is seen in steadily decreasing loss values.

Practice

1. What does the loss.backward() function do in PyTorch?

easy

A. It computes gradients of the loss with respect to model parameters.

B. It updates the model weights directly.

C. It calculates the loss value from predictions and targets.

D. It initializes the model parameters.

Solution

1. Step 1: Understand the role of loss.backward()

   This function computes gradients needed for updating model weights by backpropagation.

2. Step 2: Differentiate from other functions

   Updating weights is done by the optimizer, not loss.backward(). Calculating loss or initializing parameters are separate steps.

3. Final Answer:

   It computes gradients of the loss with respect to model parameters. -> Option A

4. Quick Check:

   Backward pass = compute gradients [OK]

Hint: Remember: backward() computes gradients, optimizer updates weights [OK]

Common Mistakes:

• Confusing backward() with optimizer step
• Thinking backward() calculates loss value
• Assuming backward() initializes parameters

2. Which of the following is the correct way to call the backward pass on a loss tensor named loss in PyTorch?

easy

A. loss.backwards()

B. loss.backward()

C. loss.backwarded()

D. loss.backward_pass()

Solution

1. Step 1: Recall PyTorch syntax for backward pass

   The correct method to compute gradients is loss.backward().

2. Step 2: Identify incorrect method names

   Other options are invalid method names and will cause errors.

3. Final Answer:

   loss.backward() -> Option B

4. Quick Check:

   Correct method name = backward() [OK]

Hint: Use exact method name: backward() with no extra letters [OK]

Common Mistakes:

• Adding extra letters or suffixes to backward
• Using plural form backwards
• Confusing method name with optimizer step

3. Consider the following PyTorch code snippet:

import torch
x = torch.tensor(2.0, requires_grad=True)
y = x ** 3
loss = y
loss.backward()
print(x.grad)

What will be the output of print(x.grad)?

medium

A. tensor(0.)

B. tensor(8.)

C. tensor(6.)

D. tensor(12.)

Solution

1. Step 1: Compute the derivative of y = x^3 at x=2

   The derivative dy/dx = 3 * x^2. At x=2, this is 3 * 4 = 12.

2. Step 2: Understand backward() computes this gradient

   Calling loss.backward() computes and stores this gradient in x.grad.

3. Final Answer:

   tensor(12.) -> Option D

4. Quick Check:

   Derivative of x^3 at 2 = 12 [OK]

Hint: Derivative of x^3 is 3x^2; plug in x=2 [OK]

Common Mistakes:

• Calculating derivative incorrectly as 8 or 6
• Forgetting to set requires_grad=True
• Not calling loss.backward() before accessing grad

4. What is the error in the following PyTorch code snippet?

import torch
x = torch.tensor(3.0, requires_grad=True)
y = x * 2
loss = y
# Missing backward call
print(x.grad)

medium

A. x.grad will be zero tensor automatically.

B. SyntaxError because backward() is missing.

C. x.grad will be None because backward() was not called.

D. RuntimeError because loss is not scalar.

Solution

1. Step 1: Identify missing backward() call

   The code does not call loss.backward(), so gradients are not computed.

2. Step 2: Understand gradient attribute behavior

   Without backward(), x.grad remains None, not zero or error.

3. Final Answer:

   x.grad will be None because backward() was not called. -> Option C

4. Quick Check:

   No backward() means no gradients [OK]

Hint: Always call loss.backward() before checking .grad [OK]

Common Mistakes:

• Expecting zero tensor instead of None
• Thinking missing backward() causes syntax error
• Assuming RuntimeError for non-scalar loss here

5. You have a model with parameters and a loss computed. You call loss.backward() twice in a row without zeroing gradients in between. What will happen to the gradients stored in the parameters?

hard

A. Gradients will accumulate, doubling the values from one backward call.

B. Gradients will reset to zero before the second backward call.

C. An error will be raised because backward() cannot be called twice.

D. Gradients will be overwritten by the second backward call.

Solution

1. Step 1: Understand gradient accumulation in PyTorch

   By default, gradients accumulate in the .grad attribute when backward() is called multiple times.

2. Step 2: Effect of calling backward() twice without zeroing

   Calling backward() twice adds gradients twice, effectively doubling them.

3. Final Answer:

   Gradients will accumulate, doubling the values from one backward call. -> Option A

4. Quick Check:

   Multiple backward calls add gradients [OK]

Hint: Call optimizer.zero_grad() to reset gradients before backward() [OK]

Common Mistakes:

• Assuming gradients reset automatically
• Expecting an error on second backward call
• Thinking gradients overwrite instead of accumulate