SSD concept in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - SSD concept

Which metric matters for SSD and WHY

SSD (Single Shot MultiBox Detector) is used for object detection. It finds objects and draws boxes around them. The key metrics are Precision, Recall, and mAP (mean Average Precision).

Precision tells us how many detected boxes are correct. Recall tells us how many real objects were found. mAP combines these to show overall detection quality across all object types and sizes.

We want high precision to avoid false alarms and high recall to find all objects. mAP summarizes this balance and is the main score to compare SSD models.

Confusion matrix for SSD detection

For object detection, confusion is about boxes:

    +----------------+----------------+
    |                | Predicted Box  |
    |                | Object / No Obj|
    +----------------+----------------+
    | Actual Object  | TP             | FN             |
    +----------------+----------------+
    | Actual No Obj  | FP             | TN             |
    +----------------+----------------+
    

TP = Correct boxes matching real objects (IoU > threshold).
FP = Boxes predicted but no real object there.
FN = Real objects missed by SSD.
TN = Background correctly ignored (usually very large, less focus).

Precision vs Recall tradeoff in SSD

If SSD is tuned to be very sure before drawing a box, it has high precision but might miss objects, so low recall. This means fewer false alarms but more missed detections.

If SSD tries to find every object, it has high recall but may draw many wrong boxes, so low precision. This means fewer misses but more false alarms.

Example: In self-driving cars, missing a pedestrian (low recall) is dangerous, so recall is more important. In photo tagging, false tags (low precision) annoy users, so precision matters more.

Good vs Bad metric values for SSD

Good SSD model:

• Precision > 0.8 (80% of predicted boxes are correct)
• Recall > 0.7 (70% of real objects detected)
• mAP > 0.75 (high overall detection quality)

Bad SSD model:

• Precision < 0.5 (many false boxes)
• Recall < 0.4 (many missed objects)
• mAP < 0.4 (poor detection performance)

Good values depend on dataset difficulty and use case but these are typical ranges.

Common pitfalls in SSD metrics

• Accuracy paradox: High accuracy can be misleading if background dominates. SSD detects many background pixels correctly but misses objects.
• IoU threshold choice: Too low threshold inflates TP, too high misses correct boxes.
• Data leakage: Training and test images overlap, inflating metrics.
• Overfitting: Model performs well on training but poorly on new images.
• Ignoring small objects: SSD may miss small objects, lowering recall.

Self-check question

Your SSD model has 98% accuracy but only 12% recall on detecting pedestrians. Is it good for production?

Answer: No. The high accuracy is misleading because most pixels are background. The very low recall means it misses almost all pedestrians, which is dangerous for real use. You need to improve recall even if accuracy drops.

Key Result

For SSD, mean Average Precision (mAP) best shows detection quality by balancing precision and recall.

Practice

1. What is the main advantage of the SSD (Single Shot MultiBox Detector) in object detection?

easy

A. It uses multiple passes to improve accuracy.

B. It detects objects in one pass, making it faster than some other methods.

C. It only detects one object per image.

D. It requires manual feature extraction before detection.

Solution

1. Step 1: Understand SSD's detection process

   SSD predicts object locations and classes in a single forward pass through the network, unlike methods that require multiple passes.

2. Step 2: Compare with other methods

   Methods like R-CNN use multiple passes, which are slower. SSD's single pass makes it faster for real-time use.

3. Final Answer:

   It detects objects in one pass, making it faster than some other methods. -> Option B

4. Quick Check:

   SSD speed = single pass detection [OK]

Hint: SSD = Single pass detection for speed [OK]

Common Mistakes:

• Thinking SSD uses multiple passes
• Believing SSD detects only one object
• Assuming manual feature extraction is needed

2. Which of the following is the correct description of SSD's output format?

easy

A. Multiple bounding boxes with class scores predicted in one pass.

B. A single class label for the whole image.

C. A heatmap showing object locations only.

D. A sequence of images with detected objects highlighted.

Solution

1. Step 1: Recall SSD output structure

   SSD outputs many bounding boxes with associated class probabilities in one forward pass.

2. Step 2: Eliminate incorrect options

   A single class label for the whole image is wrong because SSD predicts multiple objects, not one label. A heatmap showing object locations only is incorrect as SSD outputs boxes, not just heatmaps. A sequence of images with detected objects highlighted describes video or image sequence output, not SSD's prediction format.

3. Final Answer:

   Multiple bounding boxes with class scores predicted in one pass. -> Option A

4. Quick Check:

   SSD output = boxes + class scores [OK]

Hint: SSD outputs many boxes with classes at once [OK]

Common Mistakes:

• Confusing SSD output with single-label classifiers
• Thinking SSD outputs only heatmaps
• Assuming SSD outputs image sequences

3. Consider this simplified SSD output for one image:
boxes = [[0.1, 0.1, 0.4, 0.4], [0.5, 0.5, 0.9, 0.9]]
scores = [[0.9, 0.1], [0.2, 0.8]]
Here, each score list corresponds to [class0, class1]. Which box is predicted as class1 with higher confidence?

medium

A. The first box [0.1, 0.1, 0.4, 0.4]

B. Neither box predicts class1

C. Both boxes have equal confidence for class1

D. The second box [0.5, 0.5, 0.9, 0.9]

Solution

1. Step 1: Identify class1 scores for each box

   First box class1 score = 0.1, second box class1 score = 0.8.

2. Step 2: Compare scores to find higher confidence

   0.8 > 0.1, so second box has higher confidence for class1.

3. Final Answer:

   The second box [0.5, 0.5, 0.9, 0.9] -> Option D

4. Quick Check:

   Class1 highest score = 0.8 at second box [OK]

Hint: Check class1 scores, pick highest [OK]

Common Mistakes:

• Mixing class0 and class1 scores
• Ignoring score values and guessing
• Assuming equal confidence without checking

4. You have an SSD model that predicts many overlapping boxes for the same object. What common step should you add to fix this issue?

medium

A. Increase the learning rate during training.

B. Reduce the number of classes predicted.

C. Apply Non-Maximum Suppression (NMS) to remove overlapping boxes.

D. Use grayscale images instead of color.

Solution

1. Step 1: Understand overlapping box problem

   SSD often predicts multiple boxes around the same object, causing duplicates.

2. Step 2: Identify solution to remove duplicates

   Non-Maximum Suppression (NMS) filters overlapping boxes by keeping the one with highest confidence and removing others.

3. Final Answer:

   Apply Non-Maximum Suppression (NMS) to remove overlapping boxes. -> Option C

4. Quick Check:

   Overlapping boxes fix = NMS [OK]

Hint: Use NMS to clean overlapping boxes [OK]

Common Mistakes:

• Changing learning rate won't fix overlaps
• Reducing classes doesn't remove duplicates
• Switching image color doesn't affect box overlap

5. In a real-time traffic camera system using SSD, which combination of factors best balances speed and accuracy?

hard

A. Use SSD with a lightweight backbone network and moderate input image size.

B. Use SSD with a very deep backbone and very large input images.

C. Use SSD with no pre-trained weights and very small input images.

D. Use SSD with multiple passes over the image for better accuracy.

Solution

1. Step 1: Consider real-time constraints

   Real-time systems need fast processing, so a lightweight backbone helps speed up SSD.

2. Step 2: Balance input size and accuracy

   Moderate image size keeps enough detail for accuracy without slowing down processing too much.

3. Step 3: Evaluate other options

   Very deep backbones and large images slow down speed (Use SSD with a very deep backbone and very large input images.). No pre-training and very small images reduce accuracy (Use SSD with no pre-trained weights and very small input images.). Multiple passes slow down speed (Use SSD with multiple passes over the image for better accuracy.).

4. Final Answer:

   Use SSD with a lightweight backbone network and moderate input image size. -> Option A

5. Quick Check:

   Speed + accuracy = lightweight backbone + moderate size [OK]

Hint: Light backbone + moderate size = best speed-accuracy [OK]

Common Mistakes:

• Choosing very deep networks for real-time
• Using very small images losing accuracy
• Running multiple passes slows system