Custom metrics in TensorFlow - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to define a custom metric that calculates accuracy.

TensorFlow

import tensorflow as tf

def custom_accuracy(y_true, y_pred):
    correct = tf.equal(tf.argmax(y_true, axis=1), tf.argmax(y_pred, axis=1))
    return tf.reduce_mean(tf.cast(correct, [1]))

Drag options to blanks, or click blank then click option'

Atf.bool

Btf.int32

Ctf.float32

Dtf.string

Attempts:

3 left

💡 Hint

Common Mistakes

Casting to tf.int32 instead of tf.float32 causes incorrect averaging.

Returning booleans directly without casting causes errors.

The custom metric converts boolean correct predictions to float32 before averaging to get accuracy.

2fill in blank

medium

Complete the code to create a custom metric class that tracks mean squared error.

TensorFlow

import tensorflow as tf

class CustomMSE(tf.keras.metrics.Metric):
    def __init__(self, name='custom_mse', **kwargs):
        super().__init__(name=name, **kwargs)
        self.sum_squared_error = self.add_weight(name='sse', initializer='zeros')
        self.count = self.add_weight(name='count', initializer='zeros')

    def update_state(self, y_true, y_pred, sample_weight=None):
        error = tf.square(y_true - y_pred)
        self.sum_squared_error.assign_add(tf.reduce_sum(error))
        self.count.assign_add(tf.cast(tf.size(y_true), [1]))

    def result(self):
        return self.sum_squared_error / self.count

Drag options to blanks, or click blank then click option'

Atf.int64

Btf.float64

Ctf.int32

Dtf.float32

Attempts:

3 left

💡 Hint

Common Mistakes

Using integer types for count causes integer division and wrong results.

Not casting tf.size output causes type mismatch errors.

The count variable must be a floating point type to allow division in the result method.

3fill in blank

hard

Fix the error in the custom metric update_state method by completing the blank.

TensorFlow

def update_state(self, y_true, y_pred, sample_weight=None):
    error = tf.abs(y_true - y_pred)
    if sample_weight is not None:
        error = error * sample_weight
    self.total.assign_add(tf.reduce_sum(error))
    self.count.assign_add(tf.cast(tf.shape(y_true)[0], [1]))

Drag options to blanks, or click blank then click option'

Atf.float32

Btf.string

Ctf.bool

Dtf.int32

Attempts:

3 left

💡 Hint

Common Mistakes

Casting to tf.int32 causes integer division errors.

Not casting causes type mismatch errors.

Counting samples as float32 ensures correct division later in the metric calculation.

4fill in blank

hard

Fill both blanks to complete the custom metric that calculates mean absolute error with sample weights.

TensorFlow

class WeightedMAE(tf.keras.metrics.Metric):
    def __init__(self, name='weighted_mae', **kwargs):
        super().__init__(name=name, **kwargs)
        self.total_error = self.add_weight(name='total_error', initializer='zeros')
        self.total_weight = self.add_weight(name='total_weight', initializer='zeros')

    def update_state(self, y_true, y_pred, sample_weight=None):
        error = tf.abs(y_true - y_pred)
        if sample_weight is not None:
            error = error * sample_weight
            weight = sample_weight
        else:
            weight = tf.ones_like(error)
        self.total_error.assign_add(tf.reduce_sum(error))
        self.total_weight.assign_add(tf.reduce_sum([1]))

    def result(self):
        return self.total_error / self.total_weight if self.total_weight > 0 else 0.0

metric = WeightedMAE()
metric.update_state(y_true, y_pred, [2])

Drag options to blanks, or click blank then click option'

Aweight

Bsample_weight

Cerror

Dy_true

Attempts:

3 left

💡 Hint

Common Mistakes

Using error instead of weight for total_weight causes wrong weighting.

Passing error instead of sample_weight to update_state causes incorrect weighting.

The total weight is the sum of the weights used, and the update_state call uses the sample_weight argument.

5fill in blank

hard

Fill all three blanks to complete a custom metric that calculates precision for binary classification.

TensorFlow

class CustomPrecision(tf.keras.metrics.Metric):
    def __init__(self, name='custom_precision', **kwargs):
        super().__init__(name=name, **kwargs)
        self.true_positives = self.add_weight(name='tp', initializer='zeros')
        self.predicted_positives = self.add_weight(name='pp', initializer='zeros')

    def update_state(self, y_true, y_pred, sample_weight=None):
        y_pred_labels = tf.cast(y_pred > [1], tf.float32)
        true_positives = tf.reduce_sum(y_true * y_pred_labels)
        predicted_positives = tf.reduce_sum(y_pred_labels)
        self.true_positives.assign_add(true_positives)
        self.predicted_positives.assign_add(predicted_positives)

    def result(self):
        return self.true_positives / (self.predicted_positives + [2])

    def reset_states(self):
        self.true_positives.assign(0)
        self.predicted_positives.assign(0)

metric = CustomPrecision()
metric.update_state(y_true, y_pred, sample_weight=[3])

Drag options to blanks, or click blank then click option'

A0.5

B1e-7

CNone

D0.0

Attempts:

3 left

💡 Hint

Common Mistakes

Using threshold other than 0.5 causes wrong classification.

Not adding epsilon causes division by zero errors.

Passing 0 instead of None for sample_weight causes errors.

The threshold for binary classification is 0.5, a small epsilon (1e-7) avoids division by zero, and sample_weight can be None if no weighting is used.

Practice

1. What is the main purpose of a custom metric in TensorFlow model training?

easy

A. To track a special score based on true labels and predictions during training

B. To change the model's architecture automatically

C. To speed up the training process by using GPU

D. To save the model weights after each epoch

Solution

1. Step 1: Understand what metrics do

   Metrics measure how well the model performs by comparing true labels and predictions.

2. Step 2: Identify the role of custom metrics

   Custom metrics let you define your own way to measure performance during training.

3. Final Answer:

   To track a special score based on true labels and predictions during training -> Option A

4. Quick Check:

   Custom metric = special score tracking [OK]

Hint: Custom metrics measure model performance uniquely during training [OK]

Common Mistakes:

• Confusing metrics with model architecture changes
• Thinking metrics speed up training
• Believing metrics save model weights

2. Which of the following is the correct way to define a simple custom metric function in TensorFlow?

easy

A. def my_metric(y_true, y_pred): print(y_true, y_pred)

B. def my_metric(y_true, y_pred): return np.mean(np.abs(y_true - y_pred))

C. def my_metric(y_true, y_pred): return y_true + y_pred

D. def my_metric(y_true, y_pred): return tf.reduce_mean(tf.abs(y_true - y_pred))

Solution

1. Step 1: Check TensorFlow compatibility

   Custom metrics must use TensorFlow operations like tf.reduce_mean for compatibility.

2. Step 2: Identify correct function

   def my_metric(y_true, y_pred): return tf.reduce_mean(tf.abs(y_true - y_pred)) uses tf.reduce_mean and tf.abs correctly; def my_metric(y_true, y_pred): return np.mean(np.abs(y_true - y_pred)) uses numpy which is incompatible.

3. Final Answer:

   def my_metric(y_true, y_pred): return tf.reduce_mean(tf.abs(y_true - y_pred)) -> Option D

4. Quick Check:

   Use TensorFlow ops in custom metrics [OK]

Hint: Use TensorFlow ops, not numpy, inside custom metric functions [OK]

Common Mistakes:

• Using numpy functions inside metric causing errors
• Returning tensors without reduction to scalar
• Printing values instead of returning metric

3. Given the following custom metric code, what will be the output metric value for y_true = [1, 2, 3] and y_pred = [1, 2, 4]?

def custom_metric(y_true, y_pred):
    diff = tf.abs(y_true - y_pred)
    return tf.reduce_mean(diff)

medium

A. 1.0

B. 0.6667

C. 0.3333

D. 2.0

Solution

1. Step 1: Calculate absolute differences

   Differences: |1-1|=0, |2-2|=0, |3-4|=1

2. Step 2: Compute mean of differences

   Mean = (0 + 0 + 1) / 3 = 0.3333

3. Final Answer:

   0.3333 -> Option C

4. Quick Check:

   Mean absolute difference = 0.3333 [OK]

Hint: Calculate absolute differences, then average them [OK]

Common Mistakes:

• Summing differences without averaging
• Using squared difference instead of absolute
• Mixing up y_true and y_pred order

4. Identify the error in this custom metric function and choose the correct fix:

def faulty_metric(y_true, y_pred):
    return tf.reduce_sum(y_true - y_pred)

medium

A. Convert y_true and y_pred to numpy arrays first

B. Use tf.reduce_mean instead of tf.reduce_sum to get average error

C. Replace subtraction with addition

D. Add print statements to debug values

Solution

1. Step 1: Understand metric output

   Metrics should return a scalar representing average error, not total sum.

2. Step 2: Fix by averaging

   Replacing tf.reduce_sum with tf.reduce_mean returns average difference, a better metric.

3. Final Answer:

   Use tf.reduce_mean instead of tf.reduce_sum to get average error -> Option B

4. Quick Check:

   Average error metric needs reduce_mean [OK]

Hint: Use reduce_mean for average, not reduce_sum [OK]

Common Mistakes:

• Using reduce_sum causing large values
• Mixing TensorFlow and numpy operations
• Not returning a scalar metric

5. You want to create a custom metric that counts how many predictions differ from true labels by more than 0.5. Which of the following implementations is correct?

hard

A. def count_large_diff(y_true, y_pred):
diff = tf.abs(y_true - y_pred)
return tf.reduce_sum(tf.cast(diff > 0.5, tf.float32))

B. def count_large_diff(y_true, y_pred):
diff = y_true - y_pred
return tf.reduce_mean(diff > 0.5)

C. def count_large_diff(y_true, y_pred):
diff = tf.abs(y_true - y_pred)
return tf.reduce_sum(diff > 0.5)

D. def count_large_diff(y_true, y_pred):
diff = tf.abs(y_true - y_pred)
return tf.reduce_sum(tf.int32(diff > 0.5))

Solution

1. Step 1: Identify correct comparison and casting

   We must compare differences with 0.5 and cast boolean to float for summation.

2. Step 2: Check TensorFlow functions used

   def count_large_diff(y_true, y_pred): diff = tf.abs(y_true - y_pred) return tf.reduce_sum(tf.cast(diff > 0.5, tf.float32)) uses tf.abs, comparison, tf.cast to float32, and tf.reduce_sum correctly.

3. Final Answer:

   def count_large_diff(y_true, y_pred): diff = tf.abs(y_true - y_pred) return tf.reduce_sum(tf.cast(diff > 0.5, tf.float32)) -> Option A

4. Quick Check:

   Cast booleans to float before summing [OK]

Hint: Cast boolean mask to float before summing [OK]

Common Mistakes:

• Not casting boolean mask before summing
• Using reduce_mean instead of reduce_sum
• Using wrong data type cast like tf.int32