CV applications (autonomous driving, medical, retail) in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - CV applications (autonomous driving, medical, retail)

Which metric matters for CV applications and WHY

In computer vision tasks like autonomous driving, medical imaging, and retail, the choice of metric depends on the goal:

• Autonomous driving: High Recall is critical to detect all obstacles and pedestrians to avoid accidents. Precision is also important to reduce false alarms that may cause unnecessary stops.
• Medical imaging: High Recall ensures no disease cases are missed, which is vital for patient safety. Precision helps avoid false positives that can cause stress and extra tests.
• Retail (e.g., product detection): Balanced Precision and Recall matter to correctly identify products without too many mistakes, improving customer experience and inventory management.

Overall, Precision, Recall, and F1-score are key metrics. Accuracy alone can be misleading if classes are imbalanced.

Confusion Matrix Example

For a medical image classifier detecting disease (Positive) vs healthy (Negative):

      | Predicted Positive | Predicted Negative |
      |--------------------|--------------------|
      | True Positive (TP): 90  | False Negative (FN): 10 |
      | False Positive (FP): 15 | True Negative (TN): 85  |
    

Totals: TP + FP + TN + FN = 90 + 15 + 85 + 10 = 200 samples

From this matrix:

• Precision = 90 / (90 + 15) = 0.857
• Recall = 90 / (90 + 10) = 0.9
• F1-score = 2 * (0.857 * 0.9) / (0.857 + 0.9) ≈ 0.878

Precision vs Recall Tradeoff with Examples

In CV applications, improving one metric can reduce the other:

• Autonomous driving: Missing a pedestrian (low recall) can cause accidents, so recall is prioritized even if precision drops (more false alarms).
• Medical imaging: Missing a cancer case (low recall) is dangerous, so recall is critical. But too many false positives (low precision) cause unnecessary tests.
• Retail: False positives (low precision) may confuse customers, while false negatives (low recall) mean missed products. Balanced metrics improve shopping experience.

Choosing the right balance depends on the risk and cost of errors in each application.

What Good vs Bad Metric Values Look Like

• Good: Recall and precision above 0.85 in medical and autonomous driving tasks show reliable detection with few misses and false alarms.
• Bad: High accuracy (e.g., 95%) but low recall (e.g., 50%) means many positive cases are missed, which is unsafe in medical or driving contexts.
• In retail, precision or recall below 0.7 may cause poor product recognition and customer dissatisfaction.

Common Metrics Pitfalls

• Accuracy paradox: High accuracy can hide poor recall if data is imbalanced (e.g., many healthy images, few disease cases).
• Data leakage: If test images are too similar to training, metrics look better but model fails in real use.
• Overfitting indicators: Very high training metrics but low test metrics show model memorizes training data, not generalizing well.
• Ignoring class imbalance: Not using metrics like F1-score or AUC can mislead about model quality.

Self-Check Question

Your autonomous driving model has 98% accuracy but only 12% recall on detecting pedestrians. Is it good for production? Why or why not?

Answer: No, it is not good. The low recall means the model misses 88% of pedestrians, which is very dangerous. High accuracy likely comes from many non-pedestrian images. Recall is critical here to avoid accidents.

Key Result

In CV applications, recall is often most critical to avoid missing important cases, but precision and F1-score balance are also key depending on context.

Practice

1. Which of the following is a common use of computer vision in autonomous driving?

easy

A. Detecting pedestrians and other vehicles on the road

B. Managing inventory in a warehouse

C. Analyzing blood samples in a lab

D. Recommending products to online shoppers

Solution

1. Step 1: Understand autonomous driving needs

   Autonomous cars need to see and understand their surroundings to drive safely.

2. Step 2: Match computer vision tasks to driving

   Detecting pedestrians and vehicles helps the car avoid accidents and navigate roads.

3. Final Answer:

   Detecting pedestrians and other vehicles on the road -> Option A

4. Quick Check:

   Autonomous driving = detecting road objects [OK]

Hint: Autonomous driving means seeing road and traffic [OK]

Common Mistakes:

• Confusing retail or medical uses with driving
• Thinking CV only works for product tracking
• Mixing up lab analysis with driving tasks

2. Which Python library is commonly used for image processing in computer vision tasks?

easy

A. NumPy

B. Pandas

C. OpenCV

D. Matplotlib

Solution

1. Step 1: Identify libraries for image processing

   OpenCV is designed specifically for computer vision and image tasks.

2. Step 2: Compare other libraries

   NumPy handles arrays, Pandas handles tables, Matplotlib is for plotting, but OpenCV processes images.

3. Final Answer:

   OpenCV -> Option C

4. Quick Check:

   Image processing library = OpenCV [OK]

Hint: OpenCV is the go-to for CV image tasks [OK]

Common Mistakes:

• Choosing NumPy for image processing only
• Confusing Pandas with image libraries
• Picking Matplotlib which is for plotting

3. What will the following Python code output when using a pre-trained model to classify an image in a retail store?

import cv2
model = cv2.dnn.readNetFromONNX('product_classifier.onnx')
image = cv2.imread('shelf.jpg')
blob = cv2.dnn.blobFromImage(image, 1/255.0, (224,224), swapRB=True)
model.setInput(blob)
predictions = model.forward()
print(predictions.argmax())

medium

A. The raw image pixels as a list

B. The size of the input image

C. An error because the model file is missing

D. The index of the most likely product class detected

Solution

1. Step 1: Understand the code flow

   The code loads a model, prepares the image, runs prediction, and prints the class with highest score.

2. Step 2: Interpret the output

   predictions.argmax() returns the index of the class with the highest confidence, meaning the predicted product.

3. Final Answer:

   The index of the most likely product class detected -> Option D

4. Quick Check:

   Model prediction = class index [OK]

Hint: argmax gives highest scoring class index [OK]

Common Mistakes:

• Thinking it prints raw pixels
• Assuming it prints image size
• Expecting an error without checking file presence

4. A medical imaging model is not detecting tumors correctly. The code snippet is:

image = cv2.imread('scan.png')
blob = cv2.dnn.blobFromImage(image, scalefactor=1.0, size=(224,224))
model.setInput(blob)
pred = model.forward()

What is the likely issue causing poor detection?

medium

A. The image is not resized before blob creation

B. The scalefactor should normalize pixel values (e.g., 1/255.0)

C. The model input is missing color channel swap

D. The model file is not loaded

Solution

1. Step 1: Check image preprocessing

   Pixel values usually need normalization (scaling to 0-1) for good model input.

2. Step 2: Identify scalefactor problem

   Using scalefactor=1.0 keeps pixel values 0-255, which can confuse the model expecting 0-1.

3. Final Answer:

   The scalefactor should normalize pixel values (e.g., 1/255.0) -> Option B

4. Quick Check:

   Normalize pixels for model input [OK]

Hint: Normalize pixels with scalefactor 1/255.0 [OK]

Common Mistakes:

• Ignoring pixel normalization
• Assuming resizing alone fixes issues
• Forgetting color channel order matters

5. In an autonomous driving system, how can computer vision help improve safety during night driving?

hard

A. By using infrared cameras to detect pedestrians in low light

B. By increasing the car's speed automatically

C. By disabling sensors to save power

D. By only relying on GPS data

Solution

1. Step 1: Understand night driving challenges

   Low light makes it hard for normal cameras to see pedestrians and obstacles.

2. Step 2: Identify CV solution for low light

   Infrared cameras capture heat signatures, helping detect people even in darkness.

3. Final Answer:

   By using infrared cameras to detect pedestrians in low light -> Option A

4. Quick Check:

   Infrared helps see in dark [OK]

Hint: Infrared cameras detect heat at night [OK]

Common Mistakes:

• Thinking speed increase improves safety
• Disabling sensors reduces safety
• Relying only on GPS ignores vision needs