Forward pass, loss, backward, step in PyTorch - Model Metrics & Evaluation

Metrics & Evaluation - Forward pass, loss, backward, step

Which metric matters for Forward pass, loss, backward, step and WHY

The key metric here is the loss. Loss tells us how far the model's predictions are from the true answers. During the forward pass, the model makes predictions. The loss measures the error of these predictions. Then, during backward, the model learns how to improve by calculating gradients. Finally, the step updates the model to reduce the loss. Tracking loss helps us know if the model is learning well.

Confusion matrix or equivalent visualization

For classification tasks, a confusion matrix shows how many predictions were correct or wrong:

      | Predicted Positive | Predicted Negative |
      |--------------------|--------------------|
      | True Positive (TP)  | False Negative (FN) |
      | False Positive (FP) | True Negative (TN)  |
    

While the forward pass and backward step do not directly produce this matrix, the loss computed during the forward pass reflects how well the model is doing overall. Lower loss usually means better confusion matrix results.

Precision vs Recall tradeoff with concrete examples

Though precision and recall are not computed during forward/backward steps, the loss function influences them indirectly. For example:

• If the loss focuses on overall error (like cross-entropy), the model balances precision and recall.
• If the loss is weighted to penalize false negatives more, the model improves recall (finding more true positives).
• If the loss penalizes false positives more, the model improves precision (fewer wrong positive predictions).

Choosing the right loss guides the model to the desired tradeoff between precision and recall.

What "good" vs "bad" metric values look like for this use case

Good: Loss decreases steadily over training steps, showing the model is learning. For example, loss dropping from 1.0 to 0.1 means predictions are closer to true labels.

Bad: Loss stays high or fluctuates wildly, meaning the model is not learning or the learning rate is too high/low. For example, loss stuck around 1.0 or increasing means poor training.

Metrics pitfalls

• Accuracy paradox: Loss might decrease but accuracy may not improve if the model predicts common classes well but misses rare ones.
• Data leakage: If training data leaks into validation, loss looks good but model fails on new data.
• Overfitting indicators: Training loss decreases but validation loss increases, showing the model memorizes training data but does not generalize.
• Ignoring loss scale: Different loss functions have different scales; comparing raw loss values across tasks can be misleading.

Self-check question

Your model has a loss that decreases from 1.2 to 0.3 over training, but validation loss stays around 1.1. Is it good?

Answer: No, this suggests overfitting. The model learns training data well (loss 0.3) but does not generalize (validation loss 1.1). You should try regularization, more data, or early stopping.

Key Result

Loss is the key metric during forward, backward, and step; decreasing loss means the model is learning.