Softmax output layer in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output shape of a softmax layer

What is the shape of the output tensor after applying a softmax layer with 10 units to an input batch of shape (32, 100)?

TensorFlow

import tensorflow as tf
inputs = tf.random.uniform((32, 100))
softmax_layer = tf.keras.layers.Dense(10, activation='softmax')
outputs = softmax_layer(inputs)
output_shape = outputs.shape
print(output_shape)

A (10, 32)

B (32, 10)

C (32, 100)

D (100, 10)

💡 Hint

The softmax layer outputs one probability distribution per input example.

The Dense layer with 10 units outputs a tensor with shape (batch_size, 10). Since the input batch size is 32, the output shape is (32, 10).

❓ Model Choice

intermediate

Choosing the correct output layer for multi-class classification

You want to build a neural network to classify images into 5 categories. Which output layer configuration is correct?

A Dense(1, activation='sigmoid')

B Dense(1, activation='softmax')

C Dense(5, activation='sigmoid')

D Dense(5, activation='softmax')

💡 Hint

Softmax is used for multi-class classification with mutually exclusive classes.

For multi-class classification with 5 classes, the output layer should have 5 units with softmax activation to output probabilities summing to 1.

❓ Hyperparameter

advanced

Effect of temperature parameter on softmax output

In a softmax function, what is the effect of increasing the temperature parameter T > 1 on the output probabilities?

A The output probabilities do not change.

B The output probabilities become more peaked (more confident).

C The output probabilities become more uniform (less confident).

D The softmax function becomes equivalent to sigmoid.

💡 Hint

Temperature controls the sharpness of the softmax distribution.

Increasing temperature smooths the output distribution, making probabilities closer to uniform and less confident.

❓ Metrics

advanced

Correct loss function for softmax output layer

Which loss function should you use when training a model with a softmax output layer for multi-class classification?

A SparseCategoricalCrossentropy

B BinaryCrossentropy

C MeanSquaredError

D HuberLoss

💡 Hint

The loss function must match the output activation and label format.

SparseCategoricalCrossentropy is used with softmax outputs and integer labels for multi-class classification.

🔧 Debug

expert

Identifying the error in softmax output layer usage

What error will occur when compiling this model? import tensorflow as tf model = tf.keras.Sequential([ tf.keras.layers.Dense(10), tf.keras.layers.Softmax() ]) model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')

TensorFlow

import tensorflow as tf
model = tf.keras.Sequential([
  tf.keras.layers.Dense(10),
  tf.keras.layers.Softmax()
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')

A No error, model compiles successfully.

B RuntimeError: Loss function incompatible with model output.

C TypeError: Softmax layer cannot be used as a separate layer after Dense.

D ValueError: You must pass logits to SparseCategoricalCrossentropy when from_logits=True is False.

💡 Hint

Softmax can be a separate layer; loss expects probabilities by default.

The model outputs probabilities because of the Softmax layer. The loss SparseCategoricalCrossentropy expects probabilities by default (from_logits=False), so no error occurs.