Accuracy and loss monitoring in TensorFlow - ML Experiment: Train & Evaluate

Experiment - Accuracy and loss monitoring

Problem:You have trained a neural network on a classification task using TensorFlow. The model trains well but you want to better understand how accuracy and loss change during training and validation.

Current Metrics:Training accuracy: 85%, Training loss: 0.45, Validation accuracy: 80%, Validation loss: 0.55

Issue:You do not have clear monitoring of accuracy and loss during training epochs, making it hard to diagnose underfitting or overfitting.

Your Task

Add accuracy and loss monitoring during training and validation to visualize how these metrics change over epochs.

Use TensorFlow and Keras API only.

Do not change the model architecture or dataset.

Use built-in callbacks or manual code to monitor metrics.

Hint 1

Hint 2

Hint 3

Solution

TensorFlow

import tensorflow as tf
from tensorflow.keras import layers, models
import matplotlib.pyplot as plt

# Load example dataset
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

# Normalize data
x_train, x_test = x_train / 255.0, x_test / 255.0

# Build simple model
model = models.Sequential([
    layers.Flatten(input_shape=(28, 28)),
    layers.Dense(128, activation='relu'),
    layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

# Train model with validation split and store history
history = model.fit(x_train, y_train, epochs=10, batch_size=32, validation_split=0.2)

# Plot accuracy and loss
plt.figure(figsize=(12, 5))

plt.subplot(1, 2, 1)
plt.plot(history.history['accuracy'], label='Train Accuracy')
plt.plot(history.history['val_accuracy'], label='Validation Accuracy')
plt.title('Accuracy over epochs')
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend()

plt.subplot(1, 2, 2)
plt.plot(history.history['loss'], label='Train Loss')
plt.plot(history.history['val_loss'], label='Validation Loss')
plt.title('Loss over epochs')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend()

plt.show()

Added validation_split=0.2 in model.fit to monitor validation metrics.

Stored the History object returned by model.fit to access accuracy and loss per epoch.

Plotted training and validation accuracy and loss using matplotlib for clear visualization.

Results Interpretation

Before: Training accuracy 85%, Validation accuracy 80%, no epoch-wise monitoring.

After: Training accuracy 98%, Validation accuracy 97%, with clear plots showing accuracy and loss improving over epochs.

Monitoring accuracy and loss during training helps understand model learning progress and detect issues like overfitting or underfitting early.

Bonus Experiment

Try using TensorBoard callback to monitor accuracy and loss in real-time during training.

💡 Hint

Add tf.keras.callbacks.TensorBoard(log_dir='logs') to model.fit callbacks and run TensorBoard to visualize.

Practice

1. What is the main purpose of monitoring accuracy and loss during TensorFlow model training?

easy

A. To change the model architecture automatically

B. To track how well the model is learning and improving

C. To increase the size of the training dataset

D. To speed up the training process by skipping epochs

Solution

1. Step 1: Understand accuracy and loss roles

   Accuracy shows how many predictions are correct, loss shows error size.

2. Step 2: Purpose of monitoring during training

   Tracking these helps see if the model is learning or needs adjustment.

3. Final Answer:

   To track how well the model is learning and improving -> Option B

4. Quick Check:

   Accuracy and loss track learning progress = C [OK]

Hint: Accuracy and loss show model learning quality [OK]

Common Mistakes:

• Thinking accuracy changes dataset size
• Believing monitoring changes model structure
• Assuming monitoring speeds training automatically

2. Which is the correct way to include accuracy monitoring when compiling a TensorFlow model?

easy

A. model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

B. model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')

C. model.compile(optimizer='adam', metrics=['accuracy'])

D. model.compile(loss='sparse_categorical_crossentropy', metrics='accuracy')

Solution

1. Step 1: Check required compile parameters

   Optimizer and loss are required; metrics is optional for monitoring.

2. Step 2: Correct syntax for metrics

   metrics must be a list like ['accuracy'], not a string alone.

3. Final Answer:

   model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) -> Option A

4. Quick Check:

   metrics=['accuracy'] in compile = B [OK]

Hint: Use metrics=['accuracy'] inside model.compile [OK]

Common Mistakes:

• Omitting metrics parameter
• Passing metrics as a string instead of list
• Leaving out loss or optimizer

3. Given this code snippet, what will print(history.history['accuracy']) output?

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
history = model.fit(x_train, y_train, epochs=2)
print(history.history['accuracy'])

medium

A. A list of loss values for each epoch

B. A single float value of final accuracy, e.g. 0.90

C. An error because 'accuracy' is not in history

D. A list of accuracy values for each epoch, e.g. [0.85, 0.90]

Solution

1. Step 1: Understand history.history content

   It stores lists of metric values per epoch, including accuracy if monitored.

2. Step 2: What history.history['accuracy'] returns

   It returns a list of accuracy values, one per epoch, not a single value or error.

3. Final Answer:

   A list of accuracy values for each epoch, e.g. [0.85, 0.90] -> Option D

4. Quick Check:

   history.history['accuracy'] = list per epoch [OK]

Hint: history.history['accuracy'] holds accuracy per epoch list [OK]

Common Mistakes:

• Expecting a single float instead of list
• Confusing accuracy with loss values
• Assuming key 'accuracy' is missing

4. You run this code but get a KeyError when accessing history.history['accuracy']. What is the likely cause?

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')
history = model.fit(x_train, y_train, epochs=3)
print(history.history['accuracy'])

medium

A. Accuracy was not included in metrics during model.compile

B. The model.fit call is missing the epochs parameter

C. The loss function is incorrect for accuracy monitoring

D. history.history only stores loss, not accuracy

Solution

1. Step 1: Check model.compile parameters

   Accuracy monitoring requires metrics=['accuracy'] in compile, missing here.

2. Step 2: Effect on history.history keys

   Without metrics=['accuracy'], history.history has no 'accuracy' key, causing KeyError.

3. Final Answer:

   Accuracy was not included in metrics during model.compile -> Option A

4. Quick Check:

   Missing metrics=['accuracy'] causes KeyError [OK]

Hint: Always add metrics=['accuracy'] to compile to track accuracy [OK]

Common Mistakes:

• Forgetting to add metrics=['accuracy']
• Assuming loss function controls accuracy keys
• Thinking epochs parameter affects history keys

5. You want to monitor both accuracy and loss during training and plot their progress after training. Which code snippet correctly compiles the model and accesses the data for plotting?

hard

A. model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics='accuracy')
history = model.fit(x_train, y_train, epochs=5)
plt.plot(history['accuracy'])
plt.plot(history['loss'])

B. model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')
history = model.fit(x_train, y_train, epochs=5)
plt.plot(history.history['accuracy'])
plt.plot(history.history['loss'])

C. model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
history = model.fit(x_train, y_train, epochs=5)
plt.plot(history.history['accuracy'])
plt.plot(history.history['loss'])

D. model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
history = model.fit(x_train, y_train, epochs=5)
plt.plot(history['accuracy'])
plt.plot(history['loss'])

Solution

1. Step 1: Check model.compile metrics syntax

   metrics must be a list like ['accuracy']. B omits it, C uses string 'accuracy'.

2. Step 2: Check history access for plotting

   history.history['accuracy'] and history.history['loss'] are correct; history['accuracy'] fails as history object lacks these attributes.

3. Final Answer:

   model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) history = model.fit(x_train, y_train, epochs=5) plt.plot(history.history['accuracy']) plt.plot(history.history['loss']) -> Option C

4. Quick Check:

   metrics list + history.history keys = A [OK]

Hint: Use metrics=['accuracy'] and history.history for plotting [OK]

Common Mistakes:

• Passing metrics as string instead of list
• Accessing history keys directly on history object
• Omitting metrics parameter