YOLO concept in PyTorch - Model Metrics & Evaluation

Metrics & Evaluation - YOLO concept

Which metric matters for YOLO and WHY

YOLO is a model that finds objects in pictures. It needs to be good at both finding all objects (Recall) and making sure what it finds is correct (Precision). We use Precision, Recall, and mAP (mean Average Precision) to check how well YOLO works. mAP is special because it looks at how well the model finds objects at different levels of detail and overlap.

Confusion Matrix for YOLO (Object Detection)

      +-----------------------+
      |       Confusion       |
      |-----------------------|
      | True Positives (TP)   | Objects correctly found
      | False Positives (FP)  | Wrong objects found
      | False Negatives (FN)  | Objects missed
      +-----------------------+

      Total objects = TP + FN
      Total detections = TP + FP

      Precision = TP / (TP + FP)
      Recall = TP / (TP + FN)
    

For YOLO, we also check how close the predicted box is to the real box using IoU (Intersection over Union). Only predictions with IoU above a threshold count as TP.

Precision vs Recall Tradeoff in YOLO

Imagine a security camera that spots people. If YOLO has high Precision, it means it rarely says "person" when there is none (few false alarms). But it might miss some people (low Recall).

If YOLO has high Recall, it finds almost all people but might sometimes say "person" when there is none (more false alarms).

For safety, high Recall is important to not miss anyone. For counting objects in a store, high Precision is important to avoid mistakes.

Good vs Bad Metric Values for YOLO

• Good: Precision and Recall above 0.8, mAP above 0.7 means YOLO finds most objects correctly and misses few.
• Bad: Precision below 0.5 means many wrong detections. Recall below 0.5 means many objects missed. mAP below 0.4 means poor overall detection.

Common Pitfalls in YOLO Metrics

• Accuracy paradox: High accuracy can be misleading if most images have no objects. The model might just say "no object" always.
• Data leakage: Testing on images very similar to training can give too high metrics.
• Overfitting: Very high training mAP but low test mAP means model memorized training images, not learned general patterns.
• IoU threshold choice: Too low threshold inflates TP, too high makes detection too strict.

Self Check

Your YOLO model has 98% accuracy but only 12% recall on detecting cars. Is it good for production?

Answer: No. The model misses most cars (low recall). It might say "no car" often, which inflates accuracy if many images have no cars. For car detection, missing cars is bad, so recall must improve.

Key Result

YOLO's key metrics are Precision, Recall, and mAP, balancing correct detections and missed objects for reliable object detection.

Practice

1. What is the main advantage of the YOLO (You Only Look Once) model in object detection?

easy

A. It uses a very slow but highly accurate method.

B. It detects multiple objects in an image quickly in a single pass.

C. It only detects one object per image.

D. It requires multiple passes over the image to detect objects.

Solution

1. Step 1: Understand YOLO's detection method

   YOLO processes the entire image once to detect all objects, unlike methods that scan multiple times.

2. Step 2: Compare options with YOLO's feature

   It detects multiple objects in an image quickly in a single pass correctly states YOLO's speed and single-pass detection. Other options contradict this.

3. Final Answer:

   It detects multiple objects in an image quickly in a single pass. -> Option B

4. Quick Check:

   YOLO speed = single pass detection [OK]

Hint: YOLO means one pass for all objects [OK]

Common Mistakes:

• Thinking YOLO scans image multiple times
• Believing YOLO detects only one object
• Confusing YOLO with slow detection methods

2. Which of the following is the correct way to load a pretrained YOLO model in PyTorch?

easy

A. model = torch.hub.download('yolov5s')

B. model = torch.load('yolov5s.pt')

C. model = torch.yolo.load('yolov5s')

D. model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True)

Solution

1. Step 1: Recall PyTorch hub usage for YOLO

   PyTorch hub uses torch.hub.load with repo and model name to load pretrained models.

2. Step 2: Check each option

   model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True) uses correct torch.hub.load syntax with pretrained=True. model = torch.load('yolov5s.pt') loads a file but not via hub. model = torch.yolo.load('yolov5s') and D use invalid methods.

3. Final Answer:

   model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True) -> Option D

4. Quick Check:

   PyTorch hub load syntax = model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True) [OK]

Hint: Use torch.hub.load with repo and model name [OK]

Common Mistakes:

• Using torch.load without specifying hub
• Trying non-existent torch.yolo module
• Confusing download with load

3. Given this PyTorch code snippet using YOLO model:

import torch
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True)
img = torch.zeros((1, 3, 640, 640))
pred = model(img)
print(len(pred.xyxy[0]))

What does the printed number represent?

medium

A. The number of image channels

B. The batch size of the input tensor

C. The number of detected objects in the image

D. The number of model layers

Solution

1. Step 1: Understand pred.xyxy output

   pred.xyxy[0] contains bounding boxes for detected objects in the first image of the batch.

2. Step 2: len(pred.xyxy[0]) counts detected boxes

   Counting these boxes gives the number of objects detected in the image.

3. Final Answer:

   The number of detected objects in the image -> Option C

4. Quick Check:

   Length of pred.xyxy[0] = detected objects count [OK]

Hint: Length of pred.xyxy[0] = detected objects [OK]

Common Mistakes:

• Confusing output length with image channels
• Thinking it shows batch size
• Assuming it counts model layers

4. You run YOLO detection code but get an error: AttributeError: 'NoneType' object has no attribute 'xyxy'. What is the most likely cause?

medium

A. The model was not loaded correctly, so prediction returned None.

B. The input image tensor has wrong shape.

C. YOLO does not support batch size 1.

D. The torch library is outdated.

Solution

1. Step 1: Analyze the error message

   'NoneType' means prediction output is None, so accessing .xyxy fails.

2. Step 2: Identify cause of None output

   If model loading fails or is incorrect, model(img) returns None, causing this error.

3. Final Answer:

   The model was not loaded correctly, so prediction returned None. -> Option A

4. Quick Check:

   None output means model load failed [OK]

Hint: None output means model load failed [OK]

Common Mistakes:

• Blaming input shape without checking model load
• Assuming batch size 1 is unsupported
• Ignoring error message details

5. You want to detect only 'person' objects using a pretrained YOLO model in PyTorch. Which approach correctly filters the detections?

hard

A. Filter pred.xyxy[0] by checking if pred.names[class_id] == 'person' for each detected box.

B. Set model to detect only 'person' by passing classes=['person'] when loading.

C. Use a separate model trained only on persons instead of YOLO.

D. Manually crop image to only person areas before detection.

Solution

1. Step 1: Understand YOLO output structure

   YOLO outputs boxes with class IDs; pred.names maps IDs to labels like 'person'.

2. Step 2: Filter detections by class label

   Filtering pred.xyxy[0] by checking if class label equals 'person' keeps only person detections.

3. Final Answer:

   Filter pred.xyxy[0] by checking if pred.names[class_id] == 'person' for each detected box. -> Option A

4. Quick Check:

   Filter detections by class label = Filter pred.xyxy[0] by checking if pred.names[class_id] == 'person' for each detected box. [OK]

Hint: Filter detections by class label after prediction [OK]

Common Mistakes:

• Trying to limit classes at model load (not supported directly)
• Using separate model unnecessarily
• Cropping image instead of filtering results