Precision-recall curves in TensorFlow - Model Metrics & Evaluation

Metrics & Evaluation - Precision-recall curves

Which metric matters for Precision-Recall Curves and WHY

Precision-Recall curves show how well a model balances precision and recall at different thresholds. This is important when classes are imbalanced or when missing positive cases is costly. Precision tells us how many predicted positives are actually correct. Recall tells us how many real positives the model found. The curve helps us pick a threshold that fits our needs.

Confusion Matrix Example

      Actual Positive | Actual Negative
    -------------------------------------
    Predicted Positive | TP = 80    | FP = 20
    Predicted Negative | FN = 10    | TN = 90
    -------------------------------------
    Total samples = 200
    

From this matrix:

• Precision = 80 / (80 + 20) = 0.8
• Recall = 80 / (80 + 10) = 0.8889

Precision vs Recall Tradeoff with Examples

Imagine a spam filter:

• High precision: Few good emails marked as spam (low false alarms).
• High recall: Most spam emails caught (few missed spam).

If you want to avoid losing good emails, choose a threshold for high precision. If you want to catch all spam, choose high recall.

Precision-Recall curves help find this balance by showing precision and recall at many thresholds.

Good vs Bad Metric Values for Precision-Recall Curves

Good: A curve that stays near the top-right corner means both precision and recall are high across thresholds. The area under the curve (AUC-PR) close to 1.0 is excellent.

Bad: A curve near the bottom or diagonal means poor precision or recall. AUC-PR near 0.5 means the model is guessing randomly.

Common Pitfalls with Precision-Recall Metrics

• Ignoring class imbalance: Accuracy can be misleading; precision-recall curves focus on positives.
• Data leakage: Inflates precision and recall falsely.
• Overfitting: Very high precision and recall on training but poor on new data.
• Misinterpreting precision and recall: Precision is about correctness of positive predictions; recall is about coverage of actual positives.

Self Check

Your model has 98% accuracy but 12% recall on fraud cases. Is it good for production?

Answer: No. The model misses 88% of fraud cases (low recall), which is dangerous. High accuracy is misleading because fraud is rare. Focus on improving recall to catch more fraud.

Key Result

Precision-Recall curves help choose the best balance between precision and recall, especially for imbalanced data or costly errors.