Loss functions (MSELoss, CrossEntropyLoss) in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of MSELoss with simple tensors

What is the output of the following PyTorch code that calculates Mean Squared Error loss?

PyTorch

import torch
import torch.nn as nn

loss_fn = nn.MSELoss()
pred = torch.tensor([2.0, 3.0, 4.0])
target = torch.tensor([1.0, 3.0, 5.0])
loss = loss_fn(pred, target)
print(loss.item())

A 0.6666667

B 1.0

C 2.0

D 0.5

💡 Hint

Recall that MSELoss computes the average of squared differences.

MSELoss calculates the average of (pred - target)^2. Here, differences are [1, 0, -1], squares are [1, 0, 1], average is (1+0+1)/3 = 0.6666667.

❓ Predict Output

intermediate

Output of CrossEntropyLoss with logits

What is the output of this PyTorch code using CrossEntropyLoss with logits for a 3-class classification?

PyTorch

import torch
import torch.nn as nn

loss_fn = nn.CrossEntropyLoss()
logits = torch.tensor([[1.0, 2.0, 0.5]])  # batch size 1, 3 classes
labels = torch.tensor([1])  # correct class index
loss = loss_fn(logits, labels)
print(round(loss.item(), 4))

A 0.4649

B 0.4170

C 1.0986

D 0.6931

💡 Hint

CrossEntropyLoss applies softmax internally and computes negative log likelihood.

Softmax probabilities are approx [0.2311, 0.6284, 0.1404]. Negative log of probability for class 1 is -log(0.6284) ≈ 0.4649.

❓ Model Choice

advanced

Choosing loss function for regression vs classification

You want to train a model to predict house prices (a continuous value) and another model to classify images into 5 categories. Which loss functions should you choose respectively?

A MSELoss for both tasks

B CrossEntropyLoss for house prices, MSELoss for image classification

C MSELoss for house prices, CrossEntropyLoss for image classification

D CrossEntropyLoss for both tasks

💡 Hint

Think about the type of output each task produces.

Regression tasks predict continuous values, so MSELoss fits best. Classification tasks predict class probabilities, so CrossEntropyLoss is appropriate.

❓ Hyperparameter

advanced

Effect of reduction parameter in MSELoss

What is the effect of setting the 'reduction' parameter to 'sum' in PyTorch's MSELoss compared to the default 'mean'?

A Loss values are averaged over batch dimension only

B Loss values are squared twice

C Loss values are normalized by the number of classes

D Loss values are summed over all elements instead of averaged

💡 Hint

Check how reduction changes aggregation of individual losses.

'sum' adds all squared errors together, while 'mean' averages them. This changes the scale of the loss.

❓ Metrics

expert

Interpreting CrossEntropyLoss output for multi-class classification

Given a batch of 2 samples with logits and true labels, what is the CrossEntropyLoss output? Code: import torch import torch.nn as nn loss_fn = nn.CrossEntropyLoss() logits = torch.tensor([[2.0, 1.0, 0.1], [0.5, 2.5, 0.3]]) labels = torch.tensor([0, 1]) loss = loss_fn(logits, labels) print(round(loss.item(), 4))

A 0.5514

B 0.3190

C 0.4170

D 1.0986

💡 Hint

Calculate softmax probabilities for each sample and average negative log likelihood.

For sample 1: softmax probs ≈ [0.659, 0.242, 0.099], loss = -log(0.659) ≈ 0.4170. For sample 2: softmax probs ≈ [0.109, 0.803, 0.089], loss = -log(0.803) ≈ 0.2200. Average loss = (0.4170 + 0.2200)/2 = 0.3185 (rounded to 0.3190).