Why detection localizes objects in PyTorch - Why Metrics Matter

Metrics & Evaluation - Why detection localizes objects

Which metric matters and WHY

In object detection, we want to know not only if the object is found but also if it is correctly located. The key metric is Intersection over Union (IoU). IoU measures how much the predicted box overlaps with the true box. A higher IoU means better localization.

Besides IoU, Precision and Recall are important. Precision tells us how many detected boxes are correct, while Recall tells us how many true objects were found. The mean Average Precision (mAP) combines these by checking precision at different recall levels, considering localization quality.

Confusion matrix or equivalent visualization

True Positives (TP): Correctly detected and localized objects (IoU > threshold)
False Positives (FP): Detected boxes with no matching true object or low IoU
False Negatives (FN): True objects missed by the detector

Example:
TP = 80
FP = 20
FN = 15

Total objects = TP + FN = 95
Total detections = TP + FP = 100

Precision vs Recall tradeoff with examples

Imagine a security camera detecting people:

• High Precision, Low Recall: The camera only reports very sure detections. Few false alarms (FP), but it misses some people (FN). Good if false alarms are costly.
• High Recall, Low Precision: The camera reports many detections, catching almost all people but with many false alarms. Good if missing a person is worse than false alarms.

Object detection balances this tradeoff using confidence thresholds and IoU to ensure boxes are well localized and relevant.

What good vs bad metric values look like

• Good: IoU > 0.5 for most detections, Precision and Recall above 0.8, mAP near 0.8 or higher.
• Bad: Many detections with IoU < 0.3, Precision or Recall below 0.5, mAP below 0.4.

Good localization means boxes tightly cover objects. Bad means boxes are off or many objects missed.

Common pitfalls in metrics

• Ignoring IoU: Counting detections without checking overlap can inflate Precision.
• Data leakage: Testing on data seen during training gives unrealistically high metrics.
• Overfitting: Very high training metrics but poor test metrics show the model memorizes instead of generalizing.
• Threshold choice: Using too low IoU threshold can count poor boxes as correct.

Self-check question

Your object detection model has 98% accuracy but only 12% recall on detecting cars. Is it good for production?

Answer: No. High accuracy can be misleading if most images have no cars. Low recall means the model misses most cars, which is bad for safety or monitoring. You need to improve recall and localization quality.

Key Result

Object detection metrics focus on localization quality using IoU combined with precision and recall to evaluate detection correctness and completeness.

Practice

1. Why do object detection models predict bounding boxes along with class labels?

easy

A. To find the exact location of objects in the image

B. To increase the number of classes detected

C. To reduce the image size

D. To improve color accuracy

Solution

1. Step 1: Understand the role of bounding boxes

   Bounding boxes are used to mark the position and size of objects in an image.

2. Step 2: Connect bounding boxes to localization

   Predicting bounding boxes helps the model know where each object is located, not just what it is.

3. Final Answer:

   To find the exact location of objects in the image -> Option A

4. Quick Check:

   Bounding boxes = object location [OK]

Hint: Bounding boxes = object location in image [OK]

Common Mistakes:

• Thinking bounding boxes change image size
• Confusing bounding boxes with class labels
• Believing bounding boxes improve color

2. Which PyTorch output format correctly represents detection results with localization?

easy

A. (class_scores, bounding_boxes)

B. (image_tensor, class_labels)

C. (loss_value, optimizer_state)

D. (feature_maps, activation_maps)

Solution

1. Step 1: Identify detection output components

   Detection models output class scores and bounding boxes to classify and locate objects.

2. Step 2: Match output format to detection results

   The tuple (class_scores, bounding_boxes) correctly represents both classification and localization.

3. Final Answer:

   (class_scores, bounding_boxes) -> Option A

4. Quick Check:

   Detection output = (scores, boxes) [OK]

Hint: Detection outputs = class scores + bounding boxes [OK]

Common Mistakes:

• Confusing detection output with training loss
• Mixing input tensors with output results
• Using feature maps as final detection output

3. Given this PyTorch detection output code, what is the shape of boxes if batch size is 2 and max 3 detections per image?

class_scores, boxes = model(images)
print(boxes.shape)

medium

A. torch.Size([2, 4, 3])

B. torch.Size([3, 2, 4])

C. torch.Size([2, 3, 4])

D. torch.Size([4, 2, 3])

Solution

1. Step 1: Understand bounding box tensor shape

   Boxes are usually stored as (batch_size, max_detections, 4) where 4 = (x_min, y_min, x_max, y_max).

2. Step 2: Apply given values

   Batch size is 2, max detections 3, so shape is (2, 3, 4).

3. Final Answer:

   torch.Size([2, 3, 4]) -> Option C

4. Quick Check:

   Boxes shape = (batch, detections, 4) [OK]

Hint: Boxes shape = batch x detections x 4 coords [OK]

Common Mistakes:

• Swapping batch and detection dimensions
• Confusing coordinate count with detection count
• Assuming boxes shape matches class scores shape

4. You run a detection model but notice all predicted boxes are the same size and location. What is the likely cause?

medium

A. Batch size is too large

B. Bounding box regression layer is not trained properly

C. Input images are grayscale

D. Class scores are too low

Solution

1. Step 1: Identify the problem with bounding boxes

   If all boxes are identical, the model likely failed to learn how to adjust box coordinates.

2. Step 2: Connect to bounding box regression training

   The bounding box regression layer predicts box coordinates; if not trained well, outputs won't vary.

3. Final Answer:

   Bounding box regression layer is not trained properly -> Option B

4. Quick Check:

   Same boxes = bad box regression training [OK]

Hint: Same boxes mean box regression failed [OK]

Common Mistakes:

• Blaming class scores for box size issues
• Thinking grayscale images cause box errors
• Assuming batch size affects box variety

5. In a PyTorch object detection model, why is it important to predict both class scores and bounding boxes instead of only class scores?

hard

A. Because bounding boxes improve image resolution

B. Because bounding boxes reduce model size

C. Because class scores are not used in detection

D. Because class scores alone cannot tell where objects are located

Solution

1. Step 1: Understand the role of class scores

   Class scores tell what object is present but not its position in the image.

2. Step 2: Understand the role of bounding boxes

   Bounding boxes provide the exact location and size of the object, enabling localization.

3. Step 3: Combine both for full detection

   Detection requires knowing both what the object is and where it is, so both outputs are needed.

4. Final Answer:

   Because class scores alone cannot tell where objects are located -> Option D

5. Quick Check:

   Class scores = what, boxes = where [OK]

Hint: Class scores = what, boxes = where [OK]

Common Mistakes:

• Thinking bounding boxes affect image quality
• Believing class scores are unused in detection
• Assuming bounding boxes reduce model size