Custom loss functions in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of a simple custom loss function

What is the output of the following code snippet when computing the loss for y_true = [1.0, 0.0] and y_pred = [0.8, 0.3]?

TensorFlow

import tensorflow as tf

def custom_loss(y_true, y_pred):
    return tf.reduce_mean(tf.square(y_true - y_pred))

loss_value = custom_loss(tf.constant([1.0, 0.0]), tf.constant([0.8, 0.3]))
print(loss_value.numpy())

A 0.065

B 0.145

C 0.125

D 0.085

💡 Hint

Recall that mean squared error is the average of squared differences.

The squared differences are (1.0-0.8)^2 = 0.04 and (0.0-0.3)^2 = 0.09. Their mean is (0.04 + 0.09)/2 = 0.065. The printed output is 0.065.

❓ Model Choice

intermediate

Choosing a custom loss for imbalanced classification

You want to train a model on a dataset where positive examples are very rare. Which custom loss function modification helps the model focus more on positive examples?

A Use a loss function that only considers negative examples

B Use mean squared error without modification

C Ignore the positive class during loss calculation

D Add a higher weight to positive class errors in the loss function

💡 Hint

Think about how to make the model pay more attention to rare classes.

Weighting the loss more for positive class errors helps the model learn better on rare positive examples.

❓ Hyperparameter

advanced

Effect of a hyperparameter in a custom loss function

Consider a custom loss function: loss = alpha * mean_squared_error + (1 - alpha) * mean_absolute_error. What is the effect of increasing alpha from 0.3 to 0.7?

A The loss ignores mean squared error completely

B The loss focuses more on absolute errors, treating all errors equally

C The loss focuses more on squared errors, penalizing large errors more heavily

D The loss becomes invalid and causes a runtime error

💡 Hint

Recall the difference between mean squared and mean absolute errors.

Increasing alpha gives more weight to mean squared error, which penalizes large errors more than mean absolute error.

🔧 Debug

advanced

Identify the error in this custom loss function

What error does the following custom loss function code produce when used in model training? ```python def custom_loss(y_true, y_pred): return tf.reduce_sum(y_true - y_pred) ```

A SyntaxError due to missing colon

B The loss can be negative, causing training instability

C TypeError because reduce_sum is not a TensorFlow function

D No error, the loss works correctly

💡 Hint

Loss functions should be non-negative and represent error magnitude.

Using sum of differences can be negative, which is not a valid loss and can cause training issues.

🧠 Conceptual

expert

Why use custom loss functions in deep learning?

Which of the following is the best reason to create a custom loss function instead of using built-in ones?

A To tailor the loss to specific business goals or data characteristics not covered by standard losses

B Because built-in loss functions are always slower to compute

C Because custom losses always guarantee better accuracy

D To avoid using any gradient-based optimization

💡 Hint

Think about why standard losses might not fit all problems.

Custom losses allow adapting training to unique needs like special error costs or data quirks.

Practice

1. What is the main purpose of a custom loss function in TensorFlow?

easy

A. To change the model's architecture layers

B. To generate new training data from existing data

C. To speed up the training process automatically

D. To define a special way to measure how wrong the model's predictions are

Solution

1. Step 1: Understand the role of loss functions

   Loss functions measure how far the model's predictions are from the true values.

2. Step 2: Identify what custom loss functions do

   Custom loss functions let you define your own way to measure this error, which can help in special cases.

3. Final Answer:

   To define a special way to measure how wrong the model's predictions are -> Option D

4. Quick Check:

   Custom loss = custom error measure [OK]

Hint: Loss functions measure error; custom ones define your own error [OK]

Common Mistakes:

• Confusing loss function with model architecture
• Thinking loss speeds up training automatically
• Mixing loss function with data generation

2. Which of the following is the correct way to define a simple custom loss function in TensorFlow using Python?

easy

A. def custom_loss(y_true, y_pred): return tf.add(y_true, y_pred)

B. def custom_loss(y_true, y_pred): return y_true + y_pred

C. def custom_loss(y_true, y_pred): return tf.reduce_mean(tf.square(y_true - y_pred))

D. def custom_loss(y_true, y_pred): return tf.matmul(y_true, y_pred)

Solution

1. Step 1: Check the function signature

   The function must take true and predicted values as inputs.

2. Step 2: Verify the loss calculation

   Loss should measure error, e.g., mean squared error: mean of squared differences.

3. Final Answer:

   def custom_loss(y_true, y_pred): return tf.reduce_mean(tf.square(y_true - y_pred)) -> Option C

4. Quick Check:

   Loss = mean squared error formula [OK]

Hint: Loss = mean of squared differences for regression [OK]

Common Mistakes:

• Returning sum instead of mean
• Adding true and predicted values instead of difference
• Using matrix multiplication instead of error calculation

3. Given the following custom loss function, what will be the output if y_true = [2.0, 3.0] and y_pred = [1.0, 5.0]?

def custom_loss(y_true, y_pred):
    return tf.reduce_sum(tf.abs(y_true - y_pred))

medium

A. 3.0

B. 4.0

C. 5.0

D. 6.0

Solution

1. Step 1: Calculate absolute differences

   abs(2.0 - 1.0) = 1.0 and abs(3.0 - 5.0) = 2.0

2. Step 2: Sum the absolute differences

   1.0 + 2.0 = 3.0

3. Final Answer:

   3.0 -> Option A

4. Quick Check:

   Sum of absolute errors = 3.0 [OK]

Hint: Sum absolute differences for this loss [OK]

Common Mistakes:

• Confusing sum with mean
• Using squared differences instead of absolute
• Mixing up true and predicted values

4. Identify the error in this custom loss function and select the correct fix:

def custom_loss(y_true, y_pred):
    return tf.reduce_mean(y_true - y_pred)

medium

A. Change function to return y_true + y_pred

B. Use absolute difference or squared difference instead of direct subtraction

C. Remove tf.reduce_mean and return raw difference

D. Use tf.matmul instead of subtraction

Solution

1. Step 1: Understand the problem with direct subtraction

   Direct subtraction can give positive and negative values that cancel out, not measuring error properly.

2. Step 2: Correct the loss calculation

   Use absolute value or square the difference to measure error magnitude correctly.

3. Final Answer:

   Use absolute difference or squared difference instead of direct subtraction -> Option B

4. Quick Check:

   Loss must measure error magnitude, not signed difference [OK]

Hint: Loss must be positive; use abs() or square() [OK]

Common Mistakes:

• Returning signed difference causing cancellation
• Removing mean reduces loss meaning
• Using matrix multiplication wrongly

5. You want to create a custom loss function that penalizes under-predictions more than over-predictions. Which of the following custom loss functions achieves this behavior correctly?

hard

A. def custom_loss(y_true, y_pred): return tf.reduce_mean(tf.where(y_pred < y_true, 2 * (y_true - y_pred), y_pred - y_true))

B. def custom_loss(y_true, y_pred): return tf.reduce_mean(tf.abs(y_true - y_pred))

C. def custom_loss(y_true, y_pred): return tf.reduce_mean(tf.square(y_true - y_pred))

D. def custom_loss(y_true, y_pred): return tf.reduce_sum(y_pred - y_true)

Solution

1. Step 1: Understand the requirement

   We want to penalize under-predictions (y_pred < y_true) more than over-predictions.

2. Step 2: Analyze each option

   def custom_loss(y_true, y_pred): return tf.reduce_mean(tf.where(y_pred < y_true, 2 * (y_true - y_pred), y_pred - y_true)) uses tf.where to multiply the error by 2 when y_pred < y_true, increasing penalty for under-predictions. Options B and C treat errors symmetrically. def custom_loss(y_true, y_pred): return tf.reduce_sum(y_pred - y_true) sums signed differences, which is incorrect.

3. Final Answer:

   def custom_loss(y_true, y_pred): return tf.reduce_mean(tf.where(y_pred < y_true, 2 * (y_true - y_pred), y_pred - y_true)) -> Option A

4. Quick Check:

   Conditional penalty for under-prediction = def custom_loss(y_true, y_pred): return tf.reduce_mean(tf.where(y_pred < y_true, 2 * (y_true - y_pred), y_pred - y_true)) [OK]

Hint: Use tf.where to weight errors differently [OK]

Common Mistakes:

• Using symmetric loss ignoring under/over difference
• Summing signed differences causing wrong loss
• Not using conditional logic for penalty