CV applications (autonomous driving, medical, retail) in Computer Vision - Model Pipeline Trace

Model Pipeline - CV applications (autonomous driving, medical, retail)

This pipeline shows how computer vision helps in three real-life areas: autonomous driving, medical imaging, and retail. It takes images, processes them, learns patterns, and makes useful predictions like detecting objects, diseases, or products.

Data Flow - 4 Stages

1Input Images

1000 images x 256 x 256 pixels x 3 channels→Collect raw images from cameras or scanners→1000 images x 256 x 256 pixels x 3 channels

A photo of a street scene, an X-ray scan, or a store shelf image

↓

2Preprocessing

1000 images x 256 x 256 x 3→Resize, normalize pixel values, and augment images→1000 images x 224 x 224 x 3

Resized street photo with pixel values scaled between 0 and 1

↓

3Feature Extraction

1000 images x 224 x 224 x 3→Use convolutional layers to find edges, shapes, textures→1000 images x 7 x 7 x 512 features

Feature maps highlighting car edges or tumor shapes

↓

4Classification/Detection Head

1000 images x 7 x 7 x 512→Fully connected layers or detection layers predict classes or bounding boxes→1000 predictions x number_of_classes or bounding boxes

Labels like 'car', 'pedestrian', 'tumor', or product IDs with locations

Training Trace - Epoch by Epoch

Loss
1.2 |*       
0.9 | **     
0.6 |   ***  
0.3 |     ****
    +---------
     1 5 10 15 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.45	Model starts learning basic patterns, accuracy is low
5	0.7	0.70	Model improves, recognizing objects better
10	0.4	0.85	Good accuracy, model detects features well
15	0.3	0.90	Model converges, high accuracy on training data

Prediction Trace - 3 Layers

Layer 1: Input Image

Layer 2: Convolutional Layers

Layer 3: Detection Head

Model Quiz - 3 Questions

What is the main purpose of the convolutional layers in this pipeline?

A To label the objects directly

B To resize the input images

C To find important visual features like edges and shapes

D To normalize pixel values

Key Insight

Computer vision models learn to extract meaningful features from images step-by-step. This helps them detect objects or patterns in different fields like driving, medicine, and retail. Training improves the model's ability to make accurate predictions.

Practice

1. Which of the following is a common use of computer vision in autonomous driving?

easy

A. Detecting pedestrians and other vehicles on the road

B. Managing inventory in a warehouse

C. Analyzing blood samples in a lab

D. Recommending products to online shoppers

Solution

1. Step 1: Understand autonomous driving needs

   Autonomous cars need to see and understand their surroundings to drive safely.

2. Step 2: Match computer vision tasks to driving

   Detecting pedestrians and vehicles helps the car avoid accidents and navigate roads.

3. Final Answer:

   Detecting pedestrians and other vehicles on the road -> Option A

4. Quick Check:

   Autonomous driving = detecting road objects [OK]

Hint: Autonomous driving means seeing road and traffic [OK]

Common Mistakes:

• Confusing retail or medical uses with driving
• Thinking CV only works for product tracking
• Mixing up lab analysis with driving tasks

2. Which Python library is commonly used for image processing in computer vision tasks?

easy

A. NumPy

B. Pandas

C. OpenCV

D. Matplotlib

Solution

1. Step 1: Identify libraries for image processing

   OpenCV is designed specifically for computer vision and image tasks.

2. Step 2: Compare other libraries

   NumPy handles arrays, Pandas handles tables, Matplotlib is for plotting, but OpenCV processes images.

3. Final Answer:

   OpenCV -> Option C

4. Quick Check:

   Image processing library = OpenCV [OK]

Hint: OpenCV is the go-to for CV image tasks [OK]

Common Mistakes:

• Choosing NumPy for image processing only
• Confusing Pandas with image libraries
• Picking Matplotlib which is for plotting

3. What will the following Python code output when using a pre-trained model to classify an image in a retail store?

import cv2
model = cv2.dnn.readNetFromONNX('product_classifier.onnx')
image = cv2.imread('shelf.jpg')
blob = cv2.dnn.blobFromImage(image, 1/255.0, (224,224), swapRB=True)
model.setInput(blob)
predictions = model.forward()
print(predictions.argmax())

medium

A. The raw image pixels as a list

B. The size of the input image

C. An error because the model file is missing

D. The index of the most likely product class detected

Solution

1. Step 1: Understand the code flow

   The code loads a model, prepares the image, runs prediction, and prints the class with highest score.

2. Step 2: Interpret the output

   predictions.argmax() returns the index of the class with the highest confidence, meaning the predicted product.

3. Final Answer:

   The index of the most likely product class detected -> Option D

4. Quick Check:

   Model prediction = class index [OK]

Hint: argmax gives highest scoring class index [OK]

Common Mistakes:

• Thinking it prints raw pixels
• Assuming it prints image size
• Expecting an error without checking file presence

4. A medical imaging model is not detecting tumors correctly. The code snippet is:

image = cv2.imread('scan.png')
blob = cv2.dnn.blobFromImage(image, scalefactor=1.0, size=(224,224))
model.setInput(blob)
pred = model.forward()

What is the likely issue causing poor detection?

medium

A. The image is not resized before blob creation

B. The scalefactor should normalize pixel values (e.g., 1/255.0)

C. The model input is missing color channel swap

D. The model file is not loaded

Solution

1. Step 1: Check image preprocessing

   Pixel values usually need normalization (scaling to 0-1) for good model input.

2. Step 2: Identify scalefactor problem

   Using scalefactor=1.0 keeps pixel values 0-255, which can confuse the model expecting 0-1.

3. Final Answer:

   The scalefactor should normalize pixel values (e.g., 1/255.0) -> Option B

4. Quick Check:

   Normalize pixels for model input [OK]

Hint: Normalize pixels with scalefactor 1/255.0 [OK]

Common Mistakes:

• Ignoring pixel normalization
• Assuming resizing alone fixes issues
• Forgetting color channel order matters

5. In an autonomous driving system, how can computer vision help improve safety during night driving?

hard

A. By using infrared cameras to detect pedestrians in low light

B. By increasing the car's speed automatically

C. By disabling sensors to save power

D. By only relying on GPS data

Solution

1. Step 1: Understand night driving challenges

   Low light makes it hard for normal cameras to see pedestrians and obstacles.

2. Step 2: Identify CV solution for low light

   Infrared cameras capture heat signatures, helping detect people even in darkness.

3. Final Answer:

   By using infrared cameras to detect pedestrians in low light -> Option A

4. Quick Check:

   Infrared helps see in dark [OK]

Hint: Infrared cameras detect heat at night [OK]

Common Mistakes:

• Thinking speed increase improves safety
• Disabling sensors reduces safety
• Relying only on GPS ignores vision needs