Why detection localizes objects in PyTorch - Why Metrics Matter

Metrics & Evaluation - Why detection localizes objects

Which metric matters and WHY

In object detection, we want to know not only if the object is found but also if it is correctly located. The key metric is Intersection over Union (IoU). IoU measures how much the predicted box overlaps with the true box. A higher IoU means better localization.

Besides IoU, Precision and Recall are important. Precision tells us how many detected boxes are correct, while Recall tells us how many true objects were found. The mean Average Precision (mAP) combines these by checking precision at different recall levels, considering localization quality.

Confusion matrix or equivalent visualization

True Positives (TP): Correctly detected and localized objects (IoU > threshold)
False Positives (FP): Detected boxes with no matching true object or low IoU
False Negatives (FN): True objects missed by the detector

Example:
TP = 80
FP = 20
FN = 15

Total objects = TP + FN = 95
Total detections = TP + FP = 100

Precision vs Recall tradeoff with examples

Imagine a security camera detecting people:

• High Precision, Low Recall: The camera only reports very sure detections. Few false alarms (FP), but it misses some people (FN). Good if false alarms are costly.
• High Recall, Low Precision: The camera reports many detections, catching almost all people but with many false alarms. Good if missing a person is worse than false alarms.

Object detection balances this tradeoff using confidence thresholds and IoU to ensure boxes are well localized and relevant.

What good vs bad metric values look like

• Good: IoU > 0.5 for most detections, Precision and Recall above 0.8, mAP near 0.8 or higher.
• Bad: Many detections with IoU < 0.3, Precision or Recall below 0.5, mAP below 0.4.

Good localization means boxes tightly cover objects. Bad means boxes are off or many objects missed.

Common pitfalls in metrics

• Ignoring IoU: Counting detections without checking overlap can inflate Precision.
• Data leakage: Testing on data seen during training gives unrealistically high metrics.
• Overfitting: Very high training metrics but poor test metrics show the model memorizes instead of generalizing.
• Threshold choice: Using too low IoU threshold can count poor boxes as correct.

Self-check question

Your object detection model has 98% accuracy but only 12% recall on detecting cars. Is it good for production?

Answer: No. High accuracy can be misleading if most images have no cars. Low recall means the model misses most cars, which is bad for safety or monitoring. You need to improve recall and localization quality.

Key Result

Object detection metrics focus on localization quality using IoU combined with precision and recall to evaluate detection correctness and completeness.