Activation functions (ReLU, sigmoid, softmax) in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of ReLU activation on a tensor

What is the output of the following TensorFlow code applying ReLU activation?

TensorFlow

import tensorflow as tf
x = tf.constant([-3.0, 0.0, 2.0, -1.0, 5.0])
output = tf.nn.relu(x)
print(output.numpy())

A [0. 0. 0. 0. 0.]

B [-3. 0. 2. -1. 5.]

C [3. 0. 2. 1. 5.]

D [0. 0. 2. 0. 5.]

💡 Hint

ReLU sets all negative values to zero and keeps positive values unchanged.

ReLU (Rectified Linear Unit) replaces negative values with zero and leaves positive values as they are.

❓ Model Choice

intermediate

Choosing activation for binary classification output layer

Which activation function is most appropriate for the output layer of a binary classification model?

A Sigmoid

B Tanh

C ReLU

D Softmax

💡 Hint

Binary classification outputs a probability between 0 and 1.

Sigmoid outputs values between 0 and 1, suitable for binary classification probabilities.

❓ Metrics

advanced

Effect of softmax on output probabilities

Given logits [2.0, 1.0, 0.1], what is the output of applying softmax activation?

TensorFlow

import tensorflow as tf
logits = tf.constant([2.0, 1.0, 0.1])
probabilities = tf.nn.softmax(logits)
print(probabilities.numpy())

A [0.7310586 0.2689414 0.0000000]

B [0.500000 0.300000 0.200000]

C [0.65900114 0.24243297 0.09856589]

D [0.3333333 0.3333333 0.3333333]

💡 Hint

Softmax converts logits into probabilities that sum to 1.

Softmax exponentiates each logit and normalizes by the sum of all exponentials, producing probabilities.

🔧 Debug

advanced

Identifying error in sigmoid activation usage

What error will this TensorFlow code raise?

TensorFlow

import tensorflow as tf
x = tf.constant('string')
output = tf.nn.sigmoid(x)
print(output.numpy())

A TypeError: Input must be a numeric tensor

B ValueError: Cannot convert string to float

C AttributeError: 'str' object has no attribute 'numpy'

D No error, outputs sigmoid of string

💡 Hint

Sigmoid expects numeric input, not strings.

TensorFlow tries to convert string tensor to float for sigmoid but fails, raising ValueError.

🧠 Conceptual

expert

Why softmax is preferred over sigmoid for multi-class classification

Why is softmax activation preferred over sigmoid activation for the output layer in multi-class classification with mutually exclusive classes?

A Softmax outputs probabilities that sum to 1, sigmoid outputs independent probabilities that do not sum to 1

B Sigmoid is computationally more expensive than softmax

C Softmax outputs independent probabilities for each class, sigmoid outputs dependent probabilities

D Softmax can only be used with binary classification

💡 Hint

Consider the sum of output probabilities for all classes.

Softmax normalizes outputs so all class probabilities sum to 1, reflecting mutually exclusive classes. Sigmoid treats each class independently.