Model comparison in Computer Vision - Model Pipeline Trace

Model Pipeline - Model comparison

This pipeline compares two image classification models to see which one performs better on the same task. It shows how data flows, how training improves each model, and how predictions differ.

Data Flow - 6 Stages

1Input images

1000 images x 64 x 64 pixels x 3 channels→Raw images loaded for classification→1000 images x 64 x 64 pixels x 3 channels

Image of a cat with RGB colors

↓

2Preprocessing

1000 images x 64 x 64 x 3→Normalize pixel values to 0-1 range→1000 images x 64 x 64 x 3

Pixel value 128 becomes 0.5

↓

3Feature extraction Model A

1000 images x 64 x 64 x 3→Convolutional layers extract features→1000 images x 16 x 16 x 32

Edges and shapes detected in images

↓

4Feature extraction Model B

1000 images x 64 x 64 x 3→Deeper convolutional layers extract features→1000 images x 8 x 8 x 64

More detailed features detected

↓

5Flatten and Dense layers Model A

1000 images x 16 x 16 x 32→Flatten features and classify→1000 images x 10 classes

Output probabilities for 10 classes

↓

6Flatten and Dense layers Model B

1000 images x 8 x 8 x 64→Flatten features and classify→1000 images x 10 classes

Output probabilities for 10 classes

Training Trace - Epoch by Epoch

Epochs: 1  2  3
Model A Loss: 1.2-0.9-0.7
Model B Loss: 1.0-0.6-0.4
Loss decreases steadily for both models, Model B faster.

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.55	Model A starts with moderate accuracy and high loss
2	0.9	0.68	Model A improves as it learns features
3	0.7	0.75	Model A continues to improve steadily
1	1.0	0.60	Model B starts slightly better than Model A
2	0.6	0.78	Model B learns faster with deeper layers
3	0.4	0.85	Model B shows stronger performance

Prediction Trace - 5 Layers

Layer 1: Input image

Layer 2: Feature extraction Model A

Layer 3: Classification Model A

Layer 4: Feature extraction Model B

Layer 5: Classification Model B

Model Quiz - 3 Questions

Which model shows faster improvement in accuracy during training?

A Both improve equally

B Model A

C Model B

D Neither improves

Key Insight

Comparing two models side-by-side helps us see which learns faster and predicts better. Model B, with deeper layers, improves accuracy quicker and predicts with higher confidence, showing the value of more complex feature extraction in image tasks.

Practice

1. What is the main reason to compare different computer vision models on the same dataset?

easy

A. To find which model performs best for the task

B. To make the code run faster

C. To use more memory

D. To increase the dataset size

Solution

1. Step 1: Understand the purpose of model comparison

   Model comparison is done to evaluate which model gives better results on the same data.

2. Step 2: Identify the goal of comparing models

   The goal is to pick the best model for the task, not to affect code speed or data size.

3. Final Answer:

   To find which model performs best for the task -> Option A

4. Quick Check:

   Model comparison = find best model [OK]

Hint: Compare models by their results on the same data [OK]

Common Mistakes:

• Thinking comparison changes dataset size
• Confusing speed with model quality
• Assuming more memory means better model

2. Which of the following code snippets correctly compares two models' accuracy on the same test data in Python?

easy

A. acc1 = model1.fit(X_test, y_test)
acc2 = model2.fit(X_test, y_test)

B. acc1 = model1.evaluate(X_test, y_test)[1]
acc2 = model2.evaluate(X_test, y_test)[1]

C. acc1 = model1.predict(X_test)
acc2 = model2.predict(X_test)

D. acc1 = model1.score(X_train)
acc2 = model2.score(X_train)

Solution

1. Step 1: Identify correct method to get accuracy

   Using evaluate on test data returns loss and accuracy; index 1 is accuracy.

2. Step 2: Check other options for correctness

   fit trains, not evaluates; predict gives predictions, not accuracy; score needs both data and labels.

3. Final Answer:

   acc1 = model1.evaluate(X_test, y_test)[1] acc2 = model2.evaluate(X_test, y_test)[1] -> Option B

4. Quick Check:

   Use evaluate() for accuracy [OK]

Hint: Use evaluate() on test data to get accuracy [OK]

Common Mistakes:

• Using fit() instead of evaluate() for accuracy
• Using predict() output as accuracy
• Evaluating on training data instead of test data

3. Given the code below, what will be printed?

acc1 = 0.85
acc2 = 0.90
if acc1 > acc2:
    print('Model 1 is better')
else:
    print('Model 2 is better')

medium

A. Model 1 is better

B. Error: comparison not possible

C. Model 2 is better

D. No output

Solution

1. Step 1: Compare accuracy values

   acc1 is 0.85 and acc2 is 0.90, so acc1 < acc2.

2. Step 2: Follow the if-else logic

   Since acc1 > acc2 is false, the else block runs printing 'Model 2 is better'.

3. Final Answer:

   Model 2 is better -> Option C

4. Quick Check:

   0.85 < 0.90 so Model 2 wins [OK]

Hint: Compare accuracy numbers directly [OK]

Common Mistakes:

• Confusing greater than with less than
• Expecting error from simple comparison
• Ignoring else block output

4. You have two models but the code below gives an error. What is the problem?

acc1 = model1.evaluate(X_test, y_test)
acc2 = model2.evaluate(X_test, y_test)
if acc1 > acc2:
    print('Model 1 better')
else:
    print('Model 2 better')

medium

A. evaluate() returns a tuple, so direct comparison fails

B. X_test and y_test are swapped

C. Missing parentheses in print statements

D. Models are not trained yet

Solution

1. Step 1: Understand evaluate() output

   evaluate() returns a tuple (loss, accuracy), not a single number.

2. Step 2: Identify why comparison fails

   Comparing tuples directly with > causes error or unexpected behavior.

3. Final Answer:

   evaluate() returns a tuple, so direct comparison fails -> Option A

4. Quick Check:

   Compare accuracy values, not tuples [OK]

Hint: Extract accuracy from evaluate() tuple before comparing [OK]

Common Mistakes:

• Comparing full evaluate() output tuples
• Swapping test data inputs
• Assuming print syntax error

5. You want to compare three models on accuracy and speed. Which approach best helps you pick the best model?

hard

A. Use the model with smallest file size regardless of accuracy

B. Pick the model with highest accuracy only, ignoring speed

C. Choose the model with fastest training time only

D. Train all models, record accuracy and inference time, then choose the best trade-off

Solution

1. Step 1: Understand multiple criteria comparison

   Comparing models on both accuracy and speed requires measuring both metrics.

2. Step 2: Choose approach balancing accuracy and speed

   Recording accuracy and inference time helps find the best trade-off for your needs.

3. Final Answer:

   Train all models, record accuracy and inference time, then choose the best trade-off -> Option D

4. Quick Check:

   Balance accuracy and speed for best model [OK]

Hint: Measure both accuracy and speed, then compare [OK]

Common Mistakes:

• Ignoring speed when accuracy matters
• Choosing fastest training but poor accuracy
• Selecting model by size alone