Confusion matrix visualization in TensorFlow - ML Experiment: Train & Evaluate

Experiment - Confusion matrix visualization

Problem:You have trained a classification model using TensorFlow, but you want to better understand how well it predicts each class by visualizing the confusion matrix.

Current Metrics:Training accuracy: 85%, Validation accuracy: 82%

Issue:The model's overall accuracy is known, but it is unclear which classes are confused with each other, making it hard to improve the model.

Your Task

Create and visualize a confusion matrix for the validation dataset predictions to identify which classes the model confuses most.

Use TensorFlow and matplotlib only.

Do not retrain or change the model architecture.

Use the validation dataset predictions and true labels.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

TensorFlow

import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np

# Assume model, X_val, y_val are predefined
# For demonstration, create dummy data and model predictions
num_classes = 3

# Dummy true labels for validation set
y_true = np.array([0, 1, 2, 2, 1, 0, 1, 2, 0, 1])

# Dummy predicted labels from model
y_pred = np.array([0, 2, 2, 2, 1, 0, 0, 2, 0, 1])

# Compute confusion matrix
cm = tf.math.confusion_matrix(y_true, y_pred, num_classes=num_classes).numpy()

# Normalize confusion matrix by true label counts
cm_norm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]

# Plot confusion matrix
plt.figure(figsize=(6, 6))
plt.imshow(cm_norm, interpolation='nearest', cmap=plt.cm.Blues)
plt.title('Normalized Confusion Matrix')
plt.colorbar()

classes = [f'Class {i}' for i in range(num_classes)]
plt.xticks(np.arange(num_classes), classes, rotation=45)
plt.yticks(np.arange(num_classes), classes)

# Loop over data dimensions and create text annotations.
thresh = cm_norm.max() / 2.
for i in range(num_classes):
    for j in range(num_classes):
        plt.text(j, i, f'{cm[i, j]} ({cm_norm[i, j]:.2f})',
                 horizontalalignment='center',
                 color='white' if cm_norm[i, j] > thresh else 'black')

plt.ylabel('True label')
plt.xlabel('Predicted label')
plt.tight_layout()
plt.show()

Added code to compute confusion matrix using tf.math.confusion_matrix.

Normalized the confusion matrix to show proportions per true class.

Visualized the matrix with matplotlib using color coding and annotations.

Added axis labels and color bar for better understanding.

Results Interpretation

Before: Only overall accuracy was known (Training: 85%, Validation: 82%).

After: Confusion matrix visualization reveals which classes are confused, e.g., Class 1 is sometimes predicted as Class 0 or 2.

Visualizing the confusion matrix helps understand detailed model performance beyond accuracy, showing specific class prediction errors to guide improvements.

Bonus Experiment

Try creating a confusion matrix heatmap using seaborn library for enhanced visualization.

💡 Hint

Use seaborn.heatmap with annotations and a suitable color palette for clearer visuals.

Practice

1. What does a confusion matrix primarily show in machine learning?

easy

A. The size of the training dataset

B. The speed of the training process

C. The number of layers in a neural network

D. How many times each class was predicted correctly or wrongly

Solution

1. Step 1: Understand the purpose of a confusion matrix

   A confusion matrix is a table used to describe the performance of a classification model by showing correct and incorrect predictions for each class.

2. Step 2: Match the description to the options

   The description 'How many times each class was predicted correctly or wrongly' matches the purpose of a confusion matrix.

3. Final Answer:

   How many times each class was predicted correctly or wrongly -> Option D

4. Quick Check:

   Confusion matrix = correct and wrong predictions [OK]

Hint: Confusion matrix counts correct and wrong predictions per class [OK]

Common Mistakes:

• Confusing confusion matrix with training speed
• Thinking it shows model architecture details
• Assuming it shows dataset size

2. Which TensorFlow function is used to create a confusion matrix from true and predicted labels?

easy

A. tf.data.Dataset.from_tensor_slices

B. tf.keras.layers.Dense

C. tf.math.confusion_matrix

D. tf.image.resize

Solution

1. Step 1: Identify TensorFlow functions related to confusion matrix

   The function to create a confusion matrix is specifically designed to compare true and predicted labels.

2. Step 2: Match the function to the options

   tf.math.confusion_matrix is the correct TensorFlow function for this purpose, while others relate to layers, datasets, or image processing.

3. Final Answer:

   tf.math.confusion_matrix -> Option C

4. Quick Check:

   Confusion matrix function = tf.math.confusion_matrix [OK]

Hint: Use tf.math.confusion_matrix for confusion matrix in TensorFlow [OK]

Common Mistakes:

• Choosing layer or dataset functions instead
• Confusing with image processing functions
• Using non-existent TensorFlow functions

3. What is the output of this code snippet?

import tensorflow as tf
true_labels = [0, 1, 2, 2, 0]
pred_labels = [0, 2, 2, 2, 0]
cm = tf.math.confusion_matrix(true_labels, pred_labels)
print(cm.numpy())

medium

A. [[2 0 0]
[0 0 1]
[0 0 2]]

B. [[2 0 0]
[0 1 0]
[0 0 2]]

C. [[1 0 1]
[0 0 1]
[0 0 2]]

D. [[2 0 0]
[0 0 2]
[0 0 1]]

Solution

1. Step 1: Count true vs predicted labels

   For class 0: true labels are at positions 0 and 4, predicted also 0 both times -> 2 correct.
   For class 1: true label at position 1, predicted is 2 -> 0 correct, 1 predicted as 2.
   For class 2: true labels at positions 2 and 3, predicted both 2 -> 2 correct.

2. Step 2: Build confusion matrix rows

   Row 0 (true 0): predicted 0 twice -> [2,0,0]
   Row 1 (true 1): predicted 2 once -> [0,0,1]
   Row 2 (true 2): predicted 2 twice -> [0,0,2]

3. Final Answer:

   [[2 0 0] [0 0 1] [0 0 2]] -> Option A

4. Quick Check:

   Count true vs predicted labels = [[2 0 0] [0 0 1] [0 0 2]] [OK]

Hint: Count true-predicted pairs per class row-wise [OK]

Common Mistakes:

• Mixing up true and predicted label order
• Counting predicted labels as rows
• Miscounting class occurrences

4. Identify the error in this TensorFlow code for confusion matrix visualization:

import tensorflow as tf
true_labels = [0, 1, 1, 0]
pred_labels = [0, 1, 0, 0]
cm = tf.math.confusion_matrix(true_labels, pred_labels, num_classes=1)
print(cm.numpy())

medium

A. tf.math.confusion_matrix does not accept num_classes argument

B. num_classes should be 2, not 1

C. true_labels and pred_labels must be tensors, not lists

D. print(cm.numpy()) should be print(cm)

Solution

1. Step 1: Check the number of classes in labels

   True and predicted labels only contain 0 and 1, so there are 2 classes total.

2. Step 2: Verify num_classes argument

   Setting num_classes=1 is incorrect because labels include 1, which is not in [0, 1), causing a ValueError (labels out of range).

3. Final Answer:

   num_classes should be 2, not 1 -> Option B

4. Quick Check:

   num_classes must match actual classes = 2 [OK]

Hint: Set num_classes to actual number of classes in labels [OK]

Common Mistakes:

• Using wrong num_classes value
• Thinking lists are invalid inputs
• Misunderstanding print method for tensors

5. You want to visualize a confusion matrix as a heatmap using TensorFlow and Matplotlib. Which code snippet correctly creates and displays the heatmap?

hard

A. import tensorflow as tf
import matplotlib.pyplot as plt
true = [0,1,0,1]
pred = [0,0,0,1]
cm = tf.math.confusion_matrix(true, pred)
plt.imshow(cm, cmap='Blues')
plt.colorbar()
plt.show()

B. import tensorflow as tf
import matplotlib.pyplot as plt
true = [0,1,0,1]
pred = [0,0,0,1]
cm = tf.keras.metrics.ConfusionMatrix(true, pred)
plt.imshow(cm)
plt.show()

C. import tensorflow as tf
import matplotlib.pyplot as plt
true = [0,1,0,1]
pred = [0,0,0,1]
cm = tf.math.confusion_matrix(true, pred)
plt.plot(cm)
plt.show()

D. import tensorflow as tf
import matplotlib.pyplot as plt
true = [0,1,0,1]
pred = [0,0,0,1]
cm = tf.math.confusion_matrix(true, pred)
plt.bar(cm)
plt.show()

Solution

1. Step 1: Generate confusion matrix using TensorFlow

   tf.math.confusion_matrix(true, pred) correctly creates the confusion matrix tensor.

2. Step 2: Visualize matrix using Matplotlib heatmap

   plt.imshow with cmap='Blues' displays the matrix as a heatmap, plt.colorbar adds a color scale, and plt.show() renders the plot.

3. Final Answer:

   Code snippet B correctly creates and displays the heatmap -> Option A

4. Quick Check:

   Use tf.math.confusion_matrix + plt.imshow + plt.colorbar [OK]

Hint: Use plt.imshow with cmap and colorbar for heatmap [OK]

Common Mistakes:

• Using tf.keras.metrics.ConfusionMatrix (does not exist)
• Plotting confusion matrix with plt.plot or plt.bar
• Forgetting to add colorbar for scale