Loss functions (MSE, cross-entropy) in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of MSE loss calculation

What is the output of the Mean Squared Error (MSE) loss calculation for the given predictions and true values?

TensorFlow

import tensorflow as tf

y_true = tf.constant([1.0, 2.0, 3.0])
y_pred = tf.constant([1.5, 2.5, 2.0])

mse = tf.keras.losses.MeanSquaredError()
loss_value = mse(y_true, y_pred).numpy()
print(loss_value)

A 0.41666666

B 0.33333334

C 0.5

D 0.25

💡 Hint

Recall MSE is the average of squared differences between true and predicted values.

MSE is calculated as the average of (y_true - y_pred)^2. Here, differences are [-0.5, -0.5, 1.0], squared are [0.25, 0.25, 1.0], average is (0.25+0.25+1.0)/3 = 0.5.

🧠 Conceptual

intermediate

Choosing loss function for binary classification

Which loss function is most appropriate for a binary classification problem with outputs as probabilities?

A Binary Cross-Entropy

B Categorical Cross-Entropy

C Hinge Loss

D Mean Squared Error (MSE)

💡 Hint

Binary classification outputs probabilities between 0 and 1 for two classes.

Binary Cross-Entropy is designed for two-class problems with probability outputs. Categorical Cross-Entropy is for multi-class problems. MSE is less suitable for probabilities.

❓ Metrics

advanced

Interpreting cross-entropy loss value

Given a binary classification model outputting probabilities, what does a cross-entropy loss value close to 0 indicate?

A The model predictions are very close to the true labels

B The model predictions are random guesses

C The model predictions are completely wrong

D The model is overfitting

💡 Hint

Cross-entropy loss measures the difference between predicted probabilities and true labels.

A loss close to 0 means predicted probabilities match true labels well, indicating good model performance.

🔧 Debug

advanced

Error in cross-entropy loss with logits

What error will occur when using tf.keras.losses.BinaryCrossentropy(from_logits=False) on raw logits instead of probabilities?

TensorFlow

import tensorflow as tf

loss_fn = tf.keras.losses.BinaryCrossentropy(from_logits=False)
logits = tf.constant([0.0, 2.0, -1.0])
labels = tf.constant([0, 1, 0], dtype=tf.float32)
loss = loss_fn(labels, logits).numpy()
print(loss)

A TypeError because labels and logits have different types

B ValueError due to logits not being probabilities

C No error, loss computed correctly

D RuntimeWarning about invalid values in loss

💡 Hint

Check the from_logits parameter and input values expected.

Setting from_logits=False expects probabilities between 0 and 1. Passing raw logits causes ValueError because inputs are out of range.

❓ Model Choice

expert

Selecting loss function for multi-class classification with logits

You have a multi-class classification problem with 5 classes. Your model outputs raw logits (not probabilities). Which loss function and parameter setting is correct to use in TensorFlow?

A tf.keras.losses.CategoricalCrossentropy(from_logits=False)

B tf.keras.losses.BinaryCrossentropy(from_logits=True)

C tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False)

D tf.keras.losses.CategoricalCrossentropy(from_logits=True)

💡 Hint

Multi-class with logits requires categorical cross-entropy with from_logits=True.

CategoricalCrossentropy with from_logits=True correctly applies softmax internally to logits. from_logits=False expects probabilities. SparseCategoricalCrossentropy is for integer labels, not one-hot. BinaryCrossentropy is for binary tasks.

Practice

1. Which loss function is best suited for predicting continuous numbers in TensorFlow?

easy

A. Mean Squared Error (MSE)

B. Categorical Cross-Entropy

C. Binary Cross-Entropy

D. Hinge Loss

Solution

1. Step 1: Understand the type of prediction

   Continuous number prediction means the output is a real number, not categories.

2. Step 2: Match loss function to prediction type

   MSE calculates the average squared difference between predicted and true numbers, ideal for continuous values.

3. Final Answer:

   Mean Squared Error (MSE) -> Option A

4. Quick Check:

   Continuous output = MSE [OK]

Hint: Use MSE for numbers, cross-entropy for categories [OK]

Common Mistakes:

• Using cross-entropy for number prediction
• Confusing binary and categorical cross-entropy
• Choosing hinge loss for regression

2. Which of the following is the correct way to use Mean Squared Error loss in TensorFlow?

easy

A. tf.keras.losses.BinaryCrossentropy()

B. tf.losses.CrossEntropy()

C. tf.keras.losses.MeanSquaredError()

D. tf.losses.MSE()

Solution

1. Step 1: Recall TensorFlow loss function syntax

   TensorFlow uses tf.keras.losses.MeanSquaredError() for MSE loss.

2. Step 2: Check options for correct function name and module

   tf.keras.losses.MeanSquaredError() matches the correct full name and module; others are either wrong names or modules.

3. Final Answer:

   tf.keras.losses.MeanSquaredError() -> Option C

4. Quick Check:

   Correct MSE syntax = tf.keras.losses.MeanSquaredError() [OK]

Hint: Use tf.keras.losses for standard loss functions [OK]

Common Mistakes:

• Using tf.losses instead of tf.keras.losses
• Wrong function names like CrossEntropy for MSE
• Missing parentheses when creating loss object

3. What will be the output loss value when using Mean Squared Error loss in TensorFlow for predictions [2.0, 3.0] and true values [1.0, 5.0]?

medium

A. 1.5

B. 3.0

C. 4.0

D. 2.5

Solution

1. Step 1: Calculate squared errors for each prediction

   (2.0 - 1.0)^2 = 1.0, (3.0 - 5.0)^2 = 4.0

2. Step 2: Compute mean of squared errors

   (1.0 + 4.0) / 2 = 2.5

3. Step 3: Verify options

   2.5 matches 2.5, but check carefully: The question asks for output loss value from TensorFlow's MSE which returns mean, so 2.5 is correct.

4. Final Answer:

   2.5 -> Option D

5. Quick Check:

   MSE = mean squared error = 2.5 [OK]

Hint: Square errors, then average them for MSE [OK]

Common Mistakes:

• Summing errors without averaging
• Taking absolute difference instead of squared
• Mixing up predicted and true values

4. Identify the error in this TensorFlow code snippet using categorical cross-entropy loss:

model.compile(optimizer='adam', loss=tf.keras.losses.CategoricalCrossentropy, metrics=['accuracy'])

medium

A. Missing parentheses after CategoricalCrossentropy

B. Wrong optimizer name

C. Metrics should be 'loss' not 'accuracy'

D. Loss function should be a string, not an object

Solution

1. Step 1: Check loss function usage in compile

   Loss functions must be called as objects, so parentheses are needed.

2. Step 2: Identify missing parentheses

   tf.keras.losses.CategoricalCrossentropy is a class; missing () means passing the class, not an instance.

3. Final Answer:

   Missing parentheses after CategoricalCrossentropy -> Option A

4. Quick Check:

   Loss function needs () to create instance [OK]

Hint: Always add () when passing loss function classes [OK]

Common Mistakes:

• Forgetting parentheses on loss functions
• Confusing optimizer names
• Using wrong metric names

5. You have a multi-class classification problem with 4 classes. Which loss function and output layer activation should you use in TensorFlow for best results?

hard

A. Use Mean Squared Error loss with sigmoid activation

B. Use Categorical Cross-Entropy loss with softmax activation

C. Use Binary Cross-Entropy loss with softmax activation

D. Use Hinge loss with linear activation

Solution

1. Step 1: Identify problem type and output requirements

   Multi-class classification with 4 classes requires probabilities summing to 1.

2. Step 2: Match loss and activation functions

   Softmax activation outputs probabilities for each class; categorical cross-entropy measures loss for multi-class.

3. Final Answer:

   Use Categorical Cross-Entropy loss with softmax activation -> Option B

4. Quick Check:

   Multi-class = softmax + categorical cross-entropy [OK]

Hint: Softmax + categorical cross-entropy for multi-class [OK]

Common Mistakes:

• Using MSE for classification
• Using sigmoid for multi-class output
• Using binary cross-entropy for multi-class