Categorical cross-entropy loss in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of categorical cross-entropy loss calculation

What is the output value of the categorical cross-entropy loss for the given true labels and predicted probabilities?

TensorFlow

import tensorflow as tf

true_labels = tf.constant([[0, 1, 0], [1, 0, 0]], dtype=tf.float32)
predicted_probs = tf.constant([[0.1, 0.8, 0.1], [0.7, 0.2, 0.1]], dtype=tf.float32)

loss_fn = tf.keras.losses.CategoricalCrossentropy()
loss_value = loss_fn(true_labels, predicted_probs).numpy()
print(loss_value)

A 0.5108256

B 0.22314353

C 0.28990925

D 0.6931472

💡 Hint

Recall that categorical cross-entropy measures the difference between true labels and predicted probabilities using the negative log likelihood.

The categorical cross-entropy loss is calculated as the average negative log probability of the true class. For the given inputs, the loss is approximately 0.2899.

❓ Model Choice

intermediate

Choosing the correct model output activation for categorical cross-entropy

Which activation function should the model's output layer use when training with categorical cross-entropy loss on multi-class classification?

A Softmax

B Sigmoid

C ReLU

D Linear

💡 Hint

Think about how probabilities for multiple classes should sum up.

Softmax converts raw outputs into probabilities that sum to 1, which is required for categorical cross-entropy loss in multi-class classification.

❓ Hyperparameter

advanced

Effect of label smoothing on categorical cross-entropy loss

What is the main effect of applying label smoothing when using categorical cross-entropy loss during training?

A It speeds up training by reducing the number of classes.

B It prevents overfitting by making the labels less confident, distributing some probability mass to other classes.

C It increases the confidence of the model predictions by sharpening the labels.

D It converts categorical cross-entropy into mean squared error loss.

💡 Hint

Think about how smoothing changes the target labels.

Label smoothing replaces hard 0/1 labels with softer values, which reduces overconfidence and helps generalization.

🔧 Debug

advanced

Identifying the error in categorical cross-entropy loss usage

What error will this code raise when computing categorical cross-entropy loss?

TensorFlow

import tensorflow as tf

true_labels = tf.constant([1, 0, 0])
predicted_probs = tf.constant([[0.7, 0.2, 0.1]])

loss_fn = tf.keras.losses.CategoricalCrossentropy()
loss_value = loss_fn(true_labels, predicted_probs).numpy()
print(loss_value)

A RuntimeError due to invalid probability values

B TypeError because true_labels is not a float tensor

C No error, outputs a valid loss value

D ValueError due to shape mismatch between true labels and predictions

💡 Hint

Check the shapes and types of true labels and predictions.

The true labels tensor shape is (3,) but predictions shape is (1,3), causing a shape mismatch error.

🧠 Conceptual

expert

Why use categorical cross-entropy instead of sparse categorical cross-entropy?

In which scenario is categorical cross-entropy loss preferred over sparse categorical cross-entropy loss?

A When true labels are one-hot encoded vectors

B When using binary classification with two classes

C When the model output is a single scalar value

D When true labels are provided as integer class indices

💡 Hint

Consider the format of the true labels expected by each loss function.

Categorical cross-entropy expects one-hot encoded labels, while sparse categorical cross-entropy expects integer class indices.