Why automatic differentiation enables training in PyTorch - Why Metrics Matter

Metrics & Evaluation - Why automatic differentiation enables training

Which metric matters for this concept and WHY

When training a model, the key metric is the loss. Loss tells us how far the model's predictions are from the true answers. Automatic differentiation helps us find the direction to change model settings to reduce this loss. Without it, training would be very slow or impossible because we wouldn't know how to improve the model step-by-step.

Confusion matrix or equivalent visualization (ASCII)

Automatic differentiation itself does not produce a confusion matrix, but it helps minimize loss which improves metrics like accuracy, precision, and recall shown in a confusion matrix.

Confusion Matrix Example (for classification):

          Predicted
          P     N
Actual P  TP    FN
       N  FP    TN

Where:
TP = True Positives
FP = False Positives
TN = True Negatives
FN = False Negatives

Automatic differentiation helps adjust model weights to increase TP and TN, and reduce FP and FN over time.

Precision vs Recall tradeoff with concrete examples

Automatic differentiation helps find model parameters that balance precision and recall by minimizing loss. For example:

• Spam filter: High precision means fewer good emails marked as spam. Automatic differentiation adjusts weights to reduce false positives.
• Cancer detection: High recall means catching most cancer cases. Automatic differentiation helps reduce false negatives.

By computing gradients automatically, the model learns how to improve these tradeoffs efficiently.

What "good" vs "bad" metric values look like for this use case

Good training means loss steadily decreases, showing the model is learning. For example:

• Good: Loss drops from 1.0 to 0.1 over epochs, accuracy rises from 50% to 90%
• Bad: Loss stays high or jumps around, accuracy stays near random chance (e.g., 50% for binary)

Automatic differentiation enables this by providing exact gradients to update model weights correctly.

Metrics pitfalls (accuracy paradox, data leakage, overfitting indicators)

• Accuracy paradox: High accuracy can be misleading if classes are imbalanced. Automatic differentiation optimizes loss, but metric choice matters.
• Data leakage: If training data leaks info from test data, loss looks low but model fails in real use.
• Overfitting: Loss on training data drops but test loss rises. Automatic differentiation updates weights perfectly but model memorizes instead of generalizing.

Self-check: Your model has 98% accuracy but 12% recall on fraud. Is it good?

No, this is not good for fraud detection. The model misses 88% of fraud cases (low recall), which is dangerous. Automatic differentiation helped train the model, but the loss function or data might need adjustment to improve recall. High accuracy alone is misleading here.

Key Result

Automatic differentiation enables efficient training by computing exact gradients to reduce loss, improving model predictions and key metrics like accuracy, precision, and recall.