Why computer vision teaches machines to see - Why Metrics Matter

Metrics & Evaluation - Why computer vision teaches machines to see

Which metric matters for this concept and WHY

In computer vision, common tasks include recognizing objects, detecting faces, or segmenting images. The key metrics to evaluate these tasks are accuracy, precision, recall, and F1 score. These metrics tell us how well the machine "sees" and understands images. For example, precision shows how many detected objects are actually correct, while recall shows how many real objects the machine found. We use these metrics because they help us measure if the machine is making good decisions when interpreting images.

Confusion matrix or equivalent visualization (ASCII)

    Confusion Matrix Example for Object Detection:

          Predicted
          Yes    No
    Actual
    Yes   TP=80  FN=20
    No    FP=10  TN=90

    Total samples = 80 + 20 + 10 + 90 = 200

    Precision = TP / (TP + FP) = 80 / (80 + 10) = 0.89
    Recall = TP / (TP + FN) = 80 / (80 + 20) = 0.80
    F1 Score = 2 * (0.89 * 0.80) / (0.89 + 0.80) ≈ 0.84
    

Precision vs Recall tradeoff with concrete examples

Imagine a self-driving car that uses computer vision to detect pedestrians. Here, high recall is very important because missing a pedestrian (false negative) can cause accidents. So, the system should find almost all pedestrians, even if it sometimes mistakes other objects for people (lower precision).

On the other hand, a photo app that tags friends in pictures needs high precision. It should avoid tagging the wrong person (false positive) to keep users happy, even if it misses some friends (lower recall).

Balancing precision and recall depends on the goal. Computer vision models must be tuned to fit the real-life needs of their task.

What "good" vs "bad" metric values look like for this use case

Good metrics: Precision and recall above 0.85 usually mean the model sees well. For example, precision = 0.90 and recall = 0.88 means the model finds most objects and is mostly correct.

Bad metrics: Precision or recall below 0.50 means the model struggles. For example, precision = 0.40 means many false alarms, and recall = 0.45 means many objects are missed.

Accuracy alone can be misleading if the dataset is unbalanced (e.g., many images without objects). So, precision and recall give a clearer picture.

Metrics pitfalls (accuracy paradox, data leakage, overfitting indicators)

• Accuracy paradox: If most images have no objects, a model that always says "no object" can have high accuracy but is useless.
• Data leakage: If test images are too similar to training images, metrics look great but the model fails on new images.
• Overfitting: Very high training accuracy but low test accuracy means the model memorizes images instead of learning to see.

Self-check: Your model has 98% accuracy but 12% recall on detecting stop signs. Is it good?

No, it is not good. The model finds only 12% of actual stop signs, which is very low recall. Even though accuracy is high, the model misses most stop signs, which is dangerous for real driving. High recall is critical here to avoid accidents.

Key Result

Precision and recall are key to measure how well computer vision models detect and recognize objects accurately and completely.

Practice

1. What is the main goal of computer vision in machines?

easy

A. To store large amounts of data

B. To help machines understand and interpret images and videos

C. To make machines run faster

D. To improve battery life of devices

Solution

1. Step 1: Understand the purpose of computer vision

   Computer vision is about teaching machines to see and understand visual data like images and videos.

2. Step 2: Identify the correct goal

   The goal is not about speed, storage, or battery but about interpreting visual information.

3. Final Answer:

   To help machines understand and interpret images and videos -> Option B

4. Quick Check:

   Computer vision = understanding images/videos [OK]

Hint: Think: What does 'vision' mean for machines? [OK]

Common Mistakes:

• Confusing computer vision with hardware improvements
• Thinking it only stores data
• Mixing vision with battery or speed

2. Which of the following is the correct way to represent an image as data for a machine to process?

easy

A. A single number

B. A list of text descriptions

C. A matrix of pixel values

D. A sound wave

Solution

1. Step 1: Recall how images are stored digitally

   Images are stored as grids of pixels, each with color or brightness values, forming a matrix.

2. Step 2: Match the correct representation

   Only a matrix of pixel values correctly represents image data for machines.

3. Final Answer:

   A matrix of pixel values -> Option C

4. Quick Check:

   Image data = pixel matrix [OK]

Hint: Images = grids of pixels, not text or sound [OK]

Common Mistakes:

• Choosing text descriptions instead of pixel data
• Thinking images are single numbers
• Confusing images with sounds

3. Given the following Python code snippet for edge detection, what will be the output shape of edges if the input image shape is (100, 100)?

import cv2
image = cv2.imread('photo.jpg', 0)
edges = cv2.Canny(image, 100, 200)
print(edges.shape)

medium

A. (50, 50)

B. (98, 98)

C. (102, 102)

D. (100, 100)

Solution

1. Step 1: Understand Canny edge detection output size

   Canny edge detection returns an image of the same size as the input image.

2. Step 2: Check input image shape

   The input image shape is (100, 100), so the output edges will also have shape (100, 100).

3. Final Answer:

   (100, 100) -> Option D

4. Quick Check:

   Canny output shape = input shape [OK]

Hint: Edge detection keeps image size same [OK]

Common Mistakes:

• Assuming edges shrink image size
• Thinking edges enlarge image
• Confusing shape with number of edges

4. The following code is intended to convert an image to grayscale using OpenCV. What is the error?

import cv2
image = cv2.imread('photo.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
cv2.imshow('Gray Image', gray)
cv2.waitKey(0)
cv2.destroyAllWindows()

medium

A. No error, code works correctly

B. cv2.imread should include flag cv2.IMREAD_GRAYSCALE

C. cv2.cvtColor is used incorrectly

D. Missing image file path

Solution

1. Step 1: Check image reading method

   cv2.imread reads the image in color by default, which is fine for conversion.

2. Step 2: Verify color conversion usage

   cv2.cvtColor with cv2.COLOR_BGR2GRAY correctly converts color image to grayscale.

3. Step 3: Confirm display functions

   cv2.imshow, cv2.waitKey, and cv2.destroyAllWindows are used properly to show the image.

4. Final Answer:

   No error, code works correctly -> Option A

5. Quick Check:

   Correct grayscale conversion code [OK]

Hint: cv2.cvtColor with COLOR_BGR2GRAY is standard [OK]

Common Mistakes:

• Thinking cv2.imread needs grayscale flag always
• Misusing cv2.cvtColor parameters
• Forgetting to call cv2.waitKey

5. You want to teach a machine to recognize handwritten digits using computer vision. Which combination of steps is best to prepare the images before training a model?

hard

A. Convert images to grayscale, normalize pixel values, and detect edges

B. Convert images to color, increase brightness, and add noise

C. Resize images to large size, convert to text, and shuffle pixels

D. Use raw images without any processing

Solution

1. Step 1: Identify useful preprocessing steps for digit recognition

   Converting to grayscale simplifies data, normalizing scales pixel values, and edge detection highlights important features.

2. Step 2: Evaluate other options

   Color conversion and noise addition can confuse the model; resizing too large or converting to text is not helpful; raw images may have noise and irrelevant info.

3. Final Answer:

   Convert images to grayscale, normalize pixel values, and detect edges -> Option A

4. Quick Check:

   Preprocessing = grayscale + normalize + edges [OK]

Hint: Simplify images and highlight features before training [OK]

Common Mistakes:

• Using color images unnecessarily
• Adding noise that confuses model
• Skipping normalization
• Ignoring edge detection benefits