Non-maximum suppression in PyTorch - Model Metrics & Evaluation

Metrics & Evaluation - Non-maximum suppression

Which metric matters for Non-maximum suppression and WHY

Non-maximum suppression (NMS) is used in object detection to remove overlapping boxes and keep only the best ones. The key metrics to evaluate NMS are Precision and Recall. Precision tells us how many of the detected boxes are correct (not false alarms), while Recall tells us how many true objects we found. We want a balance so we keep true objects (high recall) but avoid many overlapping or wrong boxes (high precision). The Intersection over Union (IoU) threshold in NMS controls this balance by deciding when boxes overlap too much and one should be removed.

Confusion matrix for object detection with NMS (simplified)

      | Predicted Object | Predicted No Object |
      |------------------|---------------------|
      | True Positive (TP) | False Positive (FP) |
      | False Negative (FN)| True Negative (TN)  |

      TP: Correct boxes kept after NMS
      FP: Wrong boxes kept (false alarms)
      FN: True boxes removed by NMS (missed objects)
      TN: Background correctly ignored

      Total samples = TP + FP + FN + TN
    

Precision vs Recall tradeoff in Non-maximum suppression

If the IoU threshold is too low, NMS removes many boxes, increasing precision (fewer false alarms) but lowering recall (missing true objects). For example, in a face detector, too strict NMS might miss some faces (low recall).

If the IoU threshold is too high, NMS keeps many overlapping boxes, increasing recall but lowering precision (more false alarms). For example, in a car detector, too loose NMS might keep many boxes for the same car (low precision).

Choosing the right IoU threshold balances precision and recall depending on the task needs.

What "good" vs "bad" metric values look like for NMS

• Good NMS: Precision and recall both above 0.8, meaning most true objects are detected and few false boxes remain.
• Bad NMS: Precision below 0.5 means many false boxes remain, cluttering results.
• Recall below 0.5 means many true objects are missed, which is bad for safety-critical tasks like pedestrian detection.
• Very high recall but very low precision means many duplicates or false alarms.

Common pitfalls when evaluating NMS metrics

• Ignoring IoU threshold: Different IoU thresholds change precision and recall drastically, so always report the threshold used.
• Overfitting to training data: NMS tuned too tightly on training data may fail on new images.
• Confusing precision and recall: Precision is about false alarms, recall is about missed objects.
• Not considering class imbalance: If some classes are rare, metrics can be misleading.
• Using only accuracy: Accuracy is not meaningful for object detection because background dominates.

Self-check question

Your object detector with NMS has 98% accuracy but only 12% recall on pedestrians. Is it good for production? Why or why not?

Answer: No, it is not good. The high accuracy is misleading because most of the image is background (easy to classify). The very low recall means the detector misses most pedestrians, which is dangerous for applications like self-driving cars. Improving recall while keeping precision reasonable is critical.

Key Result

Non-maximum suppression balances precision and recall by removing overlapping boxes; the IoU threshold controls this tradeoff.

Practice

1. What is the main purpose of Non-maximum Suppression (NMS) in object detection using PyTorch?

easy

A. To increase the number of detected boxes

B. To resize images before detection

C. To randomly select boxes for training

D. To remove overlapping bounding boxes and keep only the best ones

Solution

1. Step 1: Understand the role of NMS

   Non-maximum suppression is used to filter out overlapping bounding boxes that represent the same object.

2. Step 2: Identify the goal of NMS

   It keeps only the bounding box with the highest confidence score among overlapping boxes to avoid duplicate detections.

3. Final Answer:

   To remove overlapping bounding boxes and keep only the best ones -> Option D

4. Quick Check:

   NMS purpose = remove overlaps, keep best [OK]

Hint: NMS filters duplicates, keeping only top scoring boxes [OK]

Common Mistakes:

• Thinking NMS increases detected boxes
• Confusing NMS with image resizing
• Assuming NMS randomly selects boxes

2. Which PyTorch function is commonly used to perform Non-maximum Suppression on bounding boxes and scores?

easy

A. torch.sort

B. torch.max

C. torch.ops.torchvision.nms

D. torch.argmax

Solution

1. Step 1: Identify the PyTorch NMS function

   PyTorch provides a dedicated function called torch.ops.torchvision.nms to perform Non-maximum Suppression.

2. Step 2: Differentiate from other functions

   torch.max, torch.sort, and torch.argmax do not perform NMS; they serve different purposes like finding max values or sorting tensors.

3. Final Answer:

   torch.ops.torchvision.nms -> Option C

4. Quick Check:

   PyTorch NMS function = torch.ops.torchvision.nms [OK]

Hint: Use torch.ops.torchvision.nms for Non-maximum Suppression in PyTorch [OK]

Common Mistakes:

• Using torch.max instead of torch.ops.torchvision.nms
• Confusing sorting functions with NMS
• Trying torch.argmax for NMS

3. Given the following PyTorch code snippet, what will be the output tensor of indices after applying NMS?

import torch
boxes = torch.tensor([[10, 10, 20, 20], [11, 11, 21, 21], [30, 30, 40, 40]], dtype=torch.float)
scores = torch.tensor([0.9, 0.8, 0.7])
iou_threshold = 0.5
indices = torch.ops.torchvision.nms(boxes, scores, iou_threshold)
print(indices.tolist())

medium

A. [0, 2]

B. [1, 2]

C. [0, 1, 2]

D. [2]

Solution

1. Step 1: Analyze box overlaps and scores

   Boxes 0 and 1 overlap with IoU > 0.5, box 2 is separate. Box 0 has higher score (0.9) than box 1 (0.8).

2. Step 2: Apply NMS logic

   NMS keeps box 0, removes box 1 due to overlap and lower score, and keeps box 2 as it does not overlap.

3. Final Answer:

   [0, 2] -> Option A

4. Quick Check:

   Keep highest score boxes without high overlap = [0, 2] [OK]

Hint: Keep highest score boxes, remove overlapping lower scores [OK]

Common Mistakes:

• Keeping all boxes ignoring overlap
• Removing box 0 instead of box 1
• Keeping only one box

4. Identify the error in this PyTorch NMS usage and select the correct fix:

import torch
boxes = torch.tensor([[0, 0, 10, 10], [1, 1, 11, 11]], dtype=torch.float)
scores = torch.tensor([0.5, 0.6])
iou_threshold = 0.3
indices = torch.nms(boxes, scores, iou_threshold)
print(indices)

medium

A. Set iou_threshold to 1.0

B. Use torch.ops.torchvision.nms instead of torch.nms

C. Change boxes dtype to int

D. Sort scores before calling nms

Solution

1. Step 1: Check function availability

   torch.nms is not a direct PyTorch function; NMS is accessed via torchvision ops as torch.ops.torchvision.nms.

2. Step 2: Correct function call

   Replacing torch.nms with torch.ops.torchvision.nms fixes the error and runs NMS correctly.

3. Final Answer:

   Use torch.ops.torchvision.nms instead of torch.nms -> Option B

4. Quick Check:

   Correct NMS call = torch.ops.torchvision.nms [OK]

Hint: Use torchvision's nms via torch.ops.torchvision.nms [OK]

Common Mistakes:

• Calling torch.nms directly
• Changing box dtype unnecessarily
• Adjusting IoU threshold without fixing function

5. You have detected multiple overlapping boxes with scores: [0.95, 0.9, 0.85, 0.6]. If you apply Non-maximum Suppression with a very low IoU threshold (e.g., 0.1), what is the expected effect on the number of boxes kept?

hard

A. Most boxes will be removed because even slight overlaps are suppressed

B. All boxes will be kept because the threshold is low

C. Only the box with the lowest score will be kept

D. Boxes will be randomly removed regardless of overlap

Solution

1. Step 1: Understand IoU threshold meaning

   A low IoU threshold means boxes with even small overlaps are considered duplicates and suppressed.

2. Step 2: Predict effect on boxes

   Because boxes overlap slightly, most will be removed except the highest scoring box, reducing the number kept.

3. Final Answer:

   Most boxes will be removed because even slight overlaps are suppressed -> Option A

4. Quick Check:

   Low IoU threshold = strict filtering = fewer boxes [OK]

Hint: Lower IoU threshold means stricter overlap removal [OK]

Common Mistakes:

• Thinking low threshold keeps more boxes
• Assuming random removal
• Believing lowest score box is kept