Loss functions (MSE, cross-entropy) in TensorFlow - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the purpose of a loss function in machine learning?

A loss function measures how well a model's predictions match the actual data. It tells the model how wrong it is so it can learn and improve.

beginner

Explain Mean Squared Error (MSE) loss function.

MSE calculates the average of the squares of the differences between predicted and actual values. It punishes bigger errors more, helping models learn precise predictions.

beginner

What type of problems is cross-entropy loss used for?

Cross-entropy loss is used for classification problems. It measures how close the predicted probabilities are to the actual class labels.

intermediate

How does cross-entropy loss handle predictions that are very wrong?

Cross-entropy loss gives a high penalty when the predicted probability for the true class is very low, encouraging the model to predict probabilities closer to the true labels.

beginner

Show a simple TensorFlow code snippet to compute MSE loss.

import tensorflow as tf

true = tf.constant([3.0, -0.5, 2.0, 7.0])
pred = tf.constant([2.5, 0.0, 2.0, 8.0])

mse = tf.reduce_mean(tf.square(pred - true))
print(f'MSE Loss: {mse.numpy()}')

Which loss function is best suited for a regression problem?

A Cross-entropy

B Categorical accuracy

C Hinge loss

D Mean Squared Error (MSE)

MSE is used for regression because it measures the average squared difference between predicted and actual continuous values.

Cross-entropy loss is mainly used for which type of machine learning task?

A Classification

B Regression

C Clustering

D Dimensionality reduction

Cross-entropy loss measures the difference between predicted probabilities and actual class labels, making it ideal for classification.

What does a high cross-entropy loss value indicate?

A Predictions are very close to true labels

B Predictions are very wrong

C Model is perfectly trained

D Loss function is not working

A high cross-entropy loss means the predicted probabilities are far from the true labels, indicating poor predictions.

In TensorFlow, which function can be used to compute MSE loss manually?

A tf.reduce_mean(tf.square(y_true - y_pred))

B tf.nn.softmax_cross_entropy_with_logits

C tf.argmax

D tf.reduce_sum(y_true + y_pred)

MSE is computed by averaging the squared differences between true and predicted values using tf.reduce_mean and tf.square.

Which loss function punishes bigger errors more strongly?

A Cross-entropy

B Mean Absolute Error

C Mean Squared Error

D Hinge loss

MSE squares the errors, so larger errors have a bigger impact on the loss value.

Describe how Mean Squared Error (MSE) and cross-entropy loss differ in their use and calculation.

Think about the type of problem and how errors are measured.

Explain why loss functions are important in training machine learning models.

Consider the role of feedback in learning.