Accuracy and loss monitoring in TensorFlow - Model Metrics & Evaluation

Metrics & Evaluation - Accuracy and loss monitoring

Which metric matters for Accuracy and Loss Monitoring and WHY

When training a model, accuracy tells us how many predictions are correct out of all tries. It is easy to understand and shows how well the model is doing overall.

Loss measures how far the model's predictions are from the true answers. Lower loss means better predictions. Loss helps guide the model to improve during training.

Both metrics together give a clear picture: accuracy shows success rate, loss shows how confident and close predictions are.

Confusion Matrix Example

      Actual \ Predicted | Positive | Negative
      -------------------|----------|---------
      Positive           |    80    |   20
      Negative           |    10    |   90
    

This matrix helps calculate accuracy and understand errors:

• True Positives (TP) = 80
• False Negatives (FN) = 20
• False Positives (FP) = 10
• True Negatives (TN) = 90

Accuracy = (TP + TN) / Total = (80 + 90) / 200 = 85%

Precision vs Recall Tradeoff

Accuracy alone can be misleading if classes are unbalanced. For example, if 95% of emails are not spam, a model that always says "not spam" has 95% accuracy but is useless.

Loss helps by showing how confident the model is, even if accuracy is high.

In some cases, you want to catch all positives (high recall), even if some mistakes happen (lower precision). In others, you want to be very sure before predicting positive (high precision).

Monitoring loss and accuracy together helps balance these needs during training.

Good vs Bad Metric Values

Good: Accuracy steadily increases and loss steadily decreases during training. For example, accuracy moving from 60% to 90%, loss dropping from 1.0 to 0.2.

Bad: Accuracy stays low or fluctuates, loss stays high or increases. This means the model is not learning well.

Also watch for loss going to zero too fast or accuracy hitting 100% early — signs of overfitting.

Common Pitfalls in Accuracy and Loss Monitoring

• Accuracy Paradox: High accuracy can hide poor performance on rare classes.
• Data Leakage: If test data leaks into training, accuracy and loss look unrealistically good.
• Overfitting: Training accuracy high but test accuracy low means model memorizes training data.
• Ignoring Loss: Only watching accuracy misses how confident or uncertain predictions are.

Self Check

Your model has 98% accuracy but only 12% recall on fraud cases. Is it good for production?

Answer: No. The model misses most fraud cases (low recall), which is dangerous. High accuracy is misleading because fraud is rare. You need to improve recall to catch more fraud.

Key Result

Accuracy shows overall correctness; loss shows prediction quality; both must be monitored to ensure good model learning.

Practice

1. What is the main purpose of monitoring accuracy and loss during TensorFlow model training?

easy

A. To change the model architecture automatically

B. To track how well the model is learning and improving

C. To increase the size of the training dataset

D. To speed up the training process by skipping epochs

Solution

1. Step 1: Understand accuracy and loss roles

   Accuracy shows how many predictions are correct, loss shows error size.

2. Step 2: Purpose of monitoring during training

   Tracking these helps see if the model is learning or needs adjustment.

3. Final Answer:

   To track how well the model is learning and improving -> Option B

4. Quick Check:

   Accuracy and loss track learning progress = C [OK]

Hint: Accuracy and loss show model learning quality [OK]

Common Mistakes:

• Thinking accuracy changes dataset size
• Believing monitoring changes model structure
• Assuming monitoring speeds training automatically

2. Which is the correct way to include accuracy monitoring when compiling a TensorFlow model?

easy

A. model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

B. model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')

C. model.compile(optimizer='adam', metrics=['accuracy'])

D. model.compile(loss='sparse_categorical_crossentropy', metrics='accuracy')

Solution

1. Step 1: Check required compile parameters

   Optimizer and loss are required; metrics is optional for monitoring.

2. Step 2: Correct syntax for metrics

   metrics must be a list like ['accuracy'], not a string alone.

3. Final Answer:

   model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) -> Option A

4. Quick Check:

   metrics=['accuracy'] in compile = B [OK]

Hint: Use metrics=['accuracy'] inside model.compile [OK]

Common Mistakes:

• Omitting metrics parameter
• Passing metrics as a string instead of list
• Leaving out loss or optimizer

3. Given this code snippet, what will print(history.history['accuracy']) output?

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
history = model.fit(x_train, y_train, epochs=2)
print(history.history['accuracy'])

medium

A. A list of loss values for each epoch

B. A single float value of final accuracy, e.g. 0.90

C. An error because 'accuracy' is not in history

D. A list of accuracy values for each epoch, e.g. [0.85, 0.90]

Solution

1. Step 1: Understand history.history content

   It stores lists of metric values per epoch, including accuracy if monitored.

2. Step 2: What history.history['accuracy'] returns

   It returns a list of accuracy values, one per epoch, not a single value or error.

3. Final Answer:

   A list of accuracy values for each epoch, e.g. [0.85, 0.90] -> Option D

4. Quick Check:

   history.history['accuracy'] = list per epoch [OK]

Hint: history.history['accuracy'] holds accuracy per epoch list [OK]

Common Mistakes:

• Expecting a single float instead of list
• Confusing accuracy with loss values
• Assuming key 'accuracy' is missing

4. You run this code but get a KeyError when accessing history.history['accuracy']. What is the likely cause?

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')
history = model.fit(x_train, y_train, epochs=3)
print(history.history['accuracy'])

medium

A. Accuracy was not included in metrics during model.compile

B. The model.fit call is missing the epochs parameter

C. The loss function is incorrect for accuracy monitoring

D. history.history only stores loss, not accuracy

Solution

1. Step 1: Check model.compile parameters

   Accuracy monitoring requires metrics=['accuracy'] in compile, missing here.

2. Step 2: Effect on history.history keys

   Without metrics=['accuracy'], history.history has no 'accuracy' key, causing KeyError.

3. Final Answer:

   Accuracy was not included in metrics during model.compile -> Option A

4. Quick Check:

   Missing metrics=['accuracy'] causes KeyError [OK]

Hint: Always add metrics=['accuracy'] to compile to track accuracy [OK]

Common Mistakes:

• Forgetting to add metrics=['accuracy']
• Assuming loss function controls accuracy keys
• Thinking epochs parameter affects history keys

5. You want to monitor both accuracy and loss during training and plot their progress after training. Which code snippet correctly compiles the model and accesses the data for plotting?

hard

A. model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics='accuracy')
history = model.fit(x_train, y_train, epochs=5)
plt.plot(history['accuracy'])
plt.plot(history['loss'])

B. model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')
history = model.fit(x_train, y_train, epochs=5)
plt.plot(history.history['accuracy'])
plt.plot(history.history['loss'])

C. model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
history = model.fit(x_train, y_train, epochs=5)
plt.plot(history.history['accuracy'])
plt.plot(history.history['loss'])

D. model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
history = model.fit(x_train, y_train, epochs=5)
plt.plot(history['accuracy'])
plt.plot(history['loss'])

Solution

1. Step 1: Check model.compile metrics syntax

   metrics must be a list like ['accuracy']. B omits it, C uses string 'accuracy'.

2. Step 2: Check history access for plotting

   history.history['accuracy'] and history.history['loss'] are correct; history['accuracy'] fails as history object lacks these attributes.

3. Final Answer:

   model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) history = model.fit(x_train, y_train, epochs=5) plt.plot(history.history['accuracy']) plt.plot(history.history['loss']) -> Option C

4. Quick Check:

   metrics list + history.history keys = A [OK]

Hint: Use metrics=['accuracy'] and history.history for plotting [OK]

Common Mistakes:

• Passing metrics as string instead of list
• Accessing history keys directly on history object
• Omitting metrics parameter