Why model persistence enables deployment in TensorFlow - Why Metrics Matter

Metrics & Evaluation - Why model persistence enables deployment

Which metric matters and WHY

When we save a model to use it later (model persistence), the key metric to watch is model consistency. This means the model's predictions and performance should stay the same before saving and after loading. This ensures the model works reliably when deployed.

Confusion matrix or equivalent visualization

Imagine a saved model tested on 100 samples before saving and after loading. The confusion matrix should be the same:

      | Predicted Positive | Predicted Negative |
      |--------------------|--------------------|
      | True Positive (TP): 40 | False Negative (FN): 10 |
      | False Positive (FP): 5 | True Negative (TN): 45 |
    

The totals add up to 100 samples. If the confusion matrix changes after loading, the model persistence failed.

Precision vs Recall tradeoff with examples

Model persistence should keep the balance between precision and recall stable. For example:

• If precision drops after loading, the model may wrongly label more negatives as positives.
• If recall drops, the model may miss more positive cases.

Both changes hurt deployment trust. Persistence must keep these metrics consistent.

What "good" vs "bad" metric values look like

Good: Model before saving and after loading has the same accuracy, precision, recall, and F1 score. For example, accuracy 90%, precision 85%, recall 80%, F1 82%.

Bad: After loading, accuracy drops to 70%, precision to 60%, recall to 50%. This means the saved model is corrupted or incompatible.

Common pitfalls in model persistence metrics

• Data leakage: If test data leaks into training, metrics look better but deployment fails.
• Overfitting: Model performs well on training but poorly after loading on new data.
• Version mismatch: Saving and loading with different TensorFlow versions can corrupt the model.
• Incomplete saving: Forgetting to save optimizer state or custom layers causes errors or metric changes.

Self-check question

Your model has 98% accuracy but 12% recall on fraud detection after loading. Is it good for deployment? Why or why not?

Answer: No, it is not good. High accuracy can be misleading if fraud cases are rare. Low recall means the model misses most fraud cases, which is dangerous. Model persistence failed to keep recall high, so deployment is risky.

Key Result

Model persistence must keep key metrics like accuracy, precision, and recall consistent before and after saving to ensure reliable deployment.