Confusion matrix analysis in TensorFlow - Model Metrics & Evaluation

Metrics & Evaluation - Confusion matrix analysis

Which metric matters and WHY

The confusion matrix helps us see how well a model predicts each class. It shows four numbers: True Positives (TP), False Positives (FP), True Negatives (TN), and False Negatives (FN). From these, we get important metrics like Precision, Recall, and Accuracy. These metrics tell us if the model is good at finding the right answers or if it makes mistakes.

Confusion Matrix Example

      Actual \ Predicted | Positive | Negative
      -------------------|----------|---------
      Positive           |    TP=50 |   FN=10
      Negative           |    FP=5  |   TN=35
    

Here, the model correctly found 50 positive cases (TP) and 35 negative cases (TN). It missed 10 positive cases (FN) and wrongly labeled 5 negatives as positive (FP).

Precision vs Recall Tradeoff

Precision tells us how many predicted positives are actually positive. Recall tells us how many actual positives the model found.

For example, in email spam detection, high precision means fewer good emails marked as spam (less annoyance). High recall means catching most spam emails.

Sometimes improving one lowers the other. We choose based on what matters more: avoiding false alarms or missing real cases.

Good vs Bad Metric Values

Good values: Precision and Recall close to 1 (like 0.9 or above) mean the model is accurate and finds most positives.

Bad values: Precision or Recall below 0.5 means many mistakes. For example, low recall means many positives are missed, which can be dangerous in medical tests.

Common Pitfalls

• Accuracy paradox: High accuracy can be misleading if classes are imbalanced (e.g., 95% accuracy but model ignores rare positive cases).
• Data leakage: Using future or test data in training inflates metrics falsely.
• Overfitting: Very high training metrics but poor test metrics means model memorizes data, not learns patterns.

Self Check

Your model has 98% accuracy but only 12% recall on fraud cases. Is it good for production?

No. The model misses 88% of fraud cases, which is risky. High accuracy is due to many non-fraud cases, but recall is critical here to catch fraud.

Key Result

Confusion matrix metrics like Precision and Recall reveal model strengths and weaknesses beyond simple accuracy.