Loss functions (MSELoss, CrossEntropyLoss) in PyTorch - Model Metrics & Evaluation

Metrics & Evaluation - Loss functions (MSELoss, CrossEntropyLoss)

Which metric matters for Loss functions and WHY

Loss functions like MSELoss and CrossEntropyLoss measure how far the model's predictions are from the true answers during training.

MSELoss (Mean Squared Error) is used for tasks where the output is a number, like predicting house prices. It calculates the average of the squared differences between predicted and actual values. Smaller loss means better predictions.

CrossEntropyLoss is used for classification tasks, like telling if an image is a cat or dog. It measures how well the predicted probabilities match the true class labels. Lower loss means the model is more confident and correct.

Tracking loss helps us know if the model is learning or not. It guides the model to improve step by step.

Confusion matrix or equivalent visualization

Loss functions do not produce a confusion matrix directly, but for classification with CrossEntropyLoss, we can look at a confusion matrix to understand errors:

      Confusion Matrix Example (3 classes):

          Predicted
          C1  C2  C3
    True C1 50   2   3
         C2  4  45   1
         C3  2   3  48

    Total samples = 50+2+3+4+45+1+2+3+48 = 158
    

This matrix helps us see where the model confuses classes, complementing the loss value.

Precision vs Recall tradeoff with examples

Loss functions like CrossEntropyLoss do not directly measure precision or recall, but they influence them by improving prediction probabilities.

For example, in a cancer detector:

• We want high recall (catch all cancer cases). CrossEntropyLoss helps by pushing the model to assign higher probabilities to true cancer cases.
• If the loss is low but recall is low, the model might be confident but missing cases.

In regression with MSELoss, the tradeoff is between fitting the data closely (low loss) and avoiding overfitting (too perfect on training data but bad on new data).

What "good" vs "bad" metric values look like for Loss functions

Good loss values:

• MSELoss: Close to 0 means predictions are very close to true values.
• CrossEntropyLoss: Close to 0 means the model predicts the correct class with high confidence.

Bad loss values:

• High MSELoss means predictions are far from true values.
• High CrossEntropyLoss means the model is uncertain or wrong about classes.

Note: Loss values depend on the problem scale and data, so compare loss over training steps or between models.

Metrics pitfalls

• Ignoring scale: MSELoss can be large if target values are large; always compare relative improvements.
• Overfitting: Loss may be very low on training data but high on new data, meaning the model memorized instead of learned.
• Data leakage: If test data leaks into training, loss looks artificially low but model fails in real use.
• Misinterpreting loss: Loss is not accuracy; a low loss does not always mean perfect predictions.

Self-check question

Your model has a training loss (CrossEntropyLoss) of 0.1 but on test data, the loss is 1.5. Is this good?

Answer: No, this means the model learned well on training data but performs poorly on new data. It likely overfitted and needs better generalization.

Key Result

Loss functions measure how far predictions are from true values; lower loss means better model learning.

Practice

1. Which loss function in PyTorch is best suited for predicting continuous values like house prices?

easy

A. torch.nn.MSELoss

B. torch.nn.CrossEntropyLoss

C. torch.nn.BCELoss

D. torch.nn.NLLLoss

Solution

1. Step 1: Understand the type of prediction

   Predicting house prices is a regression problem where output is continuous.

2. Step 2: Match loss function to problem type

   MSELoss calculates mean squared error, ideal for continuous outputs. CrossEntropyLoss is for classification, not regression.

3. Final Answer:

   torch.nn.MSELoss -> Option A

4. Quick Check:

   Regression uses MSELoss = C [OK]

Hint: Use MSELoss for continuous outputs, CrossEntropyLoss for classes [OK]

Common Mistakes:

• Choosing CrossEntropyLoss for regression tasks
• Confusing BCELoss with MSELoss
• Using NLLLoss without log probabilities

2. Which of the following is the correct way to create a CrossEntropyLoss object in PyTorch?

easy

A. loss = torch.nn.CrossEntropy()

B. loss = torch.nn.MSELoss()

C. loss = torch.nn.CrossEntropyLoss()

D. loss = torch.CrossEntropyLoss()

Solution

1. Step 1: Recall PyTorch loss function syntax

   Loss functions are in torch.nn and instantiated with parentheses.

2. Step 2: Identify correct class name and module

   CrossEntropyLoss is the correct class name under torch.nn. Options A misses 'Loss' and C misses 'nn'.

3. Final Answer:

   loss = torch.nn.CrossEntropyLoss() -> Option C

4. Quick Check:

   Correct class and module = D [OK]

Hint: Use torch.nn.CrossEntropyLoss() exactly for classification loss [OK]

Common Mistakes:

• Missing parentheses when creating loss object
• Using torch.CrossEntropyLoss() instead of torch.nn.CrossEntropyLoss()
• Confusing MSELoss with CrossEntropyLoss

3. What will be the output loss value when using torch.nn.MSELoss on predictions tensor([2.0, 3.0]) and targets tensor([1.0, 5.0])?

medium

A. 2.5

B. 3.0

C. 1.5

D. 4.0

Solution

1. Step 1: Calculate squared differences

   Differences: (2.0-1.0)=1.0, (3.0-5.0)=-2.0; squared: 1.0, 4.0

2. Step 2: Compute mean of squared differences

   Mean = (1.0 + 4.0) / 2 = 2.5

3. Final Answer:

   2.5 -> Option A

4. Quick Check:

   MSE = mean squared error = 2.5 [OK]

Hint: Square errors, then average for MSELoss output [OK]

Common Mistakes:

• Forgetting to square the differences
• Summing without averaging
• Mixing up prediction and target order

4. Given this code snippet, what is the error?

import torch
loss_fn = torch.nn.CrossEntropyLoss()
pred = torch.tensor([[0.2, 0.8], [0.6, 0.4]])
target = torch.tensor([1, 0])
loss = loss_fn(pred, target)
print(loss.item())

medium

A. Loss function should be MSELoss for this target

B. Target tensor should be one-hot encoded

C. No error, code runs correctly

D. Predictions should be raw scores (logits), not probabilities

Solution

1. Step 1: Check input requirements for CrossEntropyLoss

   CrossEntropyLoss expects raw scores (logits), not probabilities.

2. Step 2: Analyze given predictions

   Predictions are probabilities (sum to 1), so applying CrossEntropyLoss directly causes incorrect loss calculation.

3. Final Answer:

   Predictions should be raw scores (logits), not probabilities -> Option D

4. Quick Check:

   CrossEntropyLoss needs logits = A [OK]

Hint: Use logits, not probabilities, with CrossEntropyLoss [OK]

Common Mistakes:

• Passing probabilities instead of logits to CrossEntropyLoss
• Using one-hot targets instead of class indices
• Confusing MSELoss with CrossEntropyLoss for classification

5. You have a multi-class classification problem with 4 classes. Your model outputs raw scores (logits) of shape (batch_size, 4). Which loss function and target format should you use in PyTorch?

hard

A. Use torch.nn.MSELoss with target as one-hot vectors

B. Use torch.nn.CrossEntropyLoss with target as class indices (0 to 3)

C. Use torch.nn.BCELoss with target as class indices

D. Use torch.nn.CrossEntropyLoss with target as one-hot vectors

Solution

1. Step 1: Identify loss function for multi-class classification

   CrossEntropyLoss is designed for multi-class classification with logits.

2. Step 2: Determine correct target format

   CrossEntropyLoss expects target as class indices (integers), not one-hot vectors.

3. Final Answer:

   Use torch.nn.CrossEntropyLoss with target as class indices (0 to 3) -> Option B

4. Quick Check:

   CrossEntropyLoss + class indices = B [OK]

Hint: CrossEntropyLoss needs logits and integer class labels [OK]

Common Mistakes:

• Passing one-hot vectors as targets to CrossEntropyLoss
• Using MSELoss for classification
• Using BCELoss for multi-class without sigmoid