Compiling models (optimizer, loss, metrics) in TensorFlow - Model Metrics & Evaluation

Metrics & Evaluation - Compiling models (optimizer, loss, metrics)

Which metric matters for compiling models and WHY

When you compile a model, you choose an optimizer, a loss function, and metrics to watch. The loss shows how well the model is learning during training. The optimizer decides how the model changes to get better. The metrics help you understand the model's performance in ways that matter for your goal.

For example, if you want to classify images, accuracy might be a good metric. But if you want to detect rare events, precision or recall might be better. Choosing the right loss and metrics helps you know if your model is improving and if it will work well in real life.

Confusion matrix example

      Actual \ Predicted | Positive | Negative
      -------------------|----------|---------
      Positive           |    TP=50 |   FN=10
      Negative           |    FP=5  |   TN=35
    

This matrix helps calculate metrics like precision and recall, which you can add as metrics when compiling your model.

Precision vs Recall tradeoff with examples

Precision tells you how many predicted positives are actually correct. High precision means fewer false alarms.

Recall tells you how many actual positives you found. High recall means you miss fewer real cases.

Example: For a spam filter, high precision is important so you don't mark good emails as spam. For a cancer detector, high recall is important so you don't miss any cancer cases.

Good vs Bad metric values when compiling models

Good: Loss decreases steadily during training, and metrics like accuracy or F1 score improve. For example, accuracy rising from 60% to 90% means the model is learning well.

Bad: Loss stays high or jumps around, and metrics do not improve or get worse. For example, accuracy stuck at 50% (random guessing) means the model is not learning.

Common pitfalls when compiling models

• Choosing the wrong loss function for your task (e.g., using regression loss for classification) can stop learning.
• Using metrics that don't match your goal (e.g., accuracy for imbalanced data) can mislead you.
• Ignoring overfitting signs: training loss goes down but validation loss goes up.
• Data leakage: metrics look great but model sees test data during training.

Self-check question

Your model has 98% accuracy but 12% recall on fraud detection. Is it good for production? Why or why not?

Answer: No, it is not good. The model misses 88% of fraud cases (low recall), which is dangerous. Even with high accuracy, the model fails to catch most frauds, so you should improve recall.

Key Result

Choosing the right loss and metrics during model compilation ensures meaningful training and evaluation aligned with your task goals.