Model comparison in Computer Vision - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Comparing model performance metrics

You trained two image classification models. Model A has 85% accuracy and 0.35 loss. Model B has 82% accuracy and 0.28 loss. Which model is generally better?

A Model A because it has higher accuracy, which means it predicts more images correctly.

B Model B because it has lower loss, which means it fits the data better.

C Model A because accuracy is more important than loss in classification tasks.

D Model B because lower loss always means better model performance.

💡 Hint

Think about what accuracy and loss represent and which one directly shows correct predictions.

Accuracy shows the percentage of correct predictions, so higher accuracy usually means better performance. Loss measures error but can be less intuitive. Here, Model A's higher accuracy makes it better overall.

❓ Predict Output

intermediate

Output of model evaluation code

What is the printed accuracy after running this code?

Computer Vision

from sklearn.metrics import accuracy_score

y_true = [0, 1, 1, 0, 1]
y_pred = [0, 1, 0, 0, 1]
accuracy = accuracy_score(y_true, y_pred)
print(f"Accuracy: {accuracy:.2f}")

A Accuracy: 0.80

B Accuracy: 0.40

C Accuracy: 0.60

D Accuracy: 1.00

💡 Hint

Count how many predictions match the true labels and divide by total.

Out of 5 labels, 4 predictions match (positions 0,1,3,4). So accuracy = 4/5 = 0.80.

❓ Model Choice

advanced

Choosing the best model for imbalanced data

You have a dataset where 95% of images are class A and 5% are class B. You trained two models:

• Model X: 95% accuracy, but poor recall on class B.
• Model Y: 90% accuracy, but high recall on class B.

Which model is better for detecting class B?

A Model X, because it has higher overall accuracy.

B Model Y, because it detects class B better with higher recall.

C Model X, because recall is less important than accuracy.

D Model Y, because lower accuracy always means better recall.

💡 Hint

Think about which metric matters more when one class is rare.

Recall measures how many actual positives are found. For rare classes, high recall is important to catch them, so Model Y is better despite lower accuracy.

❓ Hyperparameter

advanced

Effect of batch size on model training

You train two identical neural networks on the same data but with different batch sizes: 16 and 256. Which effect is expected when using batch size 256 compared to 16?

A Training will be faster and always produce better accuracy.

B Training will be slower and always produce better accuracy.

C Training speed and accuracy will be the same regardless of batch size.

D Training will be faster per epoch but may converge to a less accurate model.

💡 Hint

Think about how batch size affects speed and model updates.

Larger batch sizes speed up training per epoch but can reduce the quality of updates, sometimes leading to worse accuracy.

🔧 Debug

expert

Identifying cause of overfitting in model training

You trained a convolutional neural network on a small dataset. Training accuracy is 98%, but validation accuracy is 60%. Which is the most likely cause?

A The model is underfitting because training accuracy is too high.

B The dataset is too large, causing the model to learn slowly.

C The model is too complex and memorizes training data, causing overfitting.

D The optimizer is not updating weights, causing poor validation accuracy.

💡 Hint

Think about what happens when training accuracy is very high but validation is low.

High training accuracy with low validation accuracy usually means the model memorizes training data but fails to generalize, indicating overfitting.

Practice

1. What is the main reason to compare different computer vision models on the same dataset?

easy

A. To find which model performs best for the task

B. To make the code run faster

C. To use more memory

D. To increase the dataset size

Solution

1. Step 1: Understand the purpose of model comparison

   Model comparison is done to evaluate which model gives better results on the same data.

2. Step 2: Identify the goal of comparing models

   The goal is to pick the best model for the task, not to affect code speed or data size.

3. Final Answer:

   To find which model performs best for the task -> Option A

4. Quick Check:

   Model comparison = find best model [OK]

Hint: Compare models by their results on the same data [OK]

Common Mistakes:

• Thinking comparison changes dataset size
• Confusing speed with model quality
• Assuming more memory means better model

2. Which of the following code snippets correctly compares two models' accuracy on the same test data in Python?

easy

A. acc1 = model1.fit(X_test, y_test)
acc2 = model2.fit(X_test, y_test)

B. acc1 = model1.evaluate(X_test, y_test)[1]
acc2 = model2.evaluate(X_test, y_test)[1]

C. acc1 = model1.predict(X_test)
acc2 = model2.predict(X_test)

D. acc1 = model1.score(X_train)
acc2 = model2.score(X_train)

Solution

1. Step 1: Identify correct method to get accuracy

   Using evaluate on test data returns loss and accuracy; index 1 is accuracy.

2. Step 2: Check other options for correctness

   fit trains, not evaluates; predict gives predictions, not accuracy; score needs both data and labels.

3. Final Answer:

   acc1 = model1.evaluate(X_test, y_test)[1] acc2 = model2.evaluate(X_test, y_test)[1] -> Option B

4. Quick Check:

   Use evaluate() for accuracy [OK]

Hint: Use evaluate() on test data to get accuracy [OK]

Common Mistakes:

• Using fit() instead of evaluate() for accuracy
• Using predict() output as accuracy
• Evaluating on training data instead of test data

3. Given the code below, what will be printed?

acc1 = 0.85
acc2 = 0.90
if acc1 > acc2:
    print('Model 1 is better')
else:
    print('Model 2 is better')

medium

A. Model 1 is better

B. Error: comparison not possible

C. Model 2 is better

D. No output

Solution

1. Step 1: Compare accuracy values

   acc1 is 0.85 and acc2 is 0.90, so acc1 < acc2.

2. Step 2: Follow the if-else logic

   Since acc1 > acc2 is false, the else block runs printing 'Model 2 is better'.

3. Final Answer:

   Model 2 is better -> Option C

4. Quick Check:

   0.85 < 0.90 so Model 2 wins [OK]

Hint: Compare accuracy numbers directly [OK]

Common Mistakes:

• Confusing greater than with less than
• Expecting error from simple comparison
• Ignoring else block output

4. You have two models but the code below gives an error. What is the problem?

acc1 = model1.evaluate(X_test, y_test)
acc2 = model2.evaluate(X_test, y_test)
if acc1 > acc2:
    print('Model 1 better')
else:
    print('Model 2 better')

medium

A. evaluate() returns a tuple, so direct comparison fails

B. X_test and y_test are swapped

C. Missing parentheses in print statements

D. Models are not trained yet

Solution

1. Step 1: Understand evaluate() output

   evaluate() returns a tuple (loss, accuracy), not a single number.

2. Step 2: Identify why comparison fails

   Comparing tuples directly with > causes error or unexpected behavior.

3. Final Answer:

   evaluate() returns a tuple, so direct comparison fails -> Option A

4. Quick Check:

   Compare accuracy values, not tuples [OK]

Hint: Extract accuracy from evaluate() tuple before comparing [OK]

Common Mistakes:

• Comparing full evaluate() output tuples
• Swapping test data inputs
• Assuming print syntax error

5. You want to compare three models on accuracy and speed. Which approach best helps you pick the best model?

hard

A. Use the model with smallest file size regardless of accuracy

B. Pick the model with highest accuracy only, ignoring speed

C. Choose the model with fastest training time only

D. Train all models, record accuracy and inference time, then choose the best trade-off

Solution

1. Step 1: Understand multiple criteria comparison

   Comparing models on both accuracy and speed requires measuring both metrics.

2. Step 2: Choose approach balancing accuracy and speed

   Recording accuracy and inference time helps find the best trade-off for your needs.

3. Final Answer:

   Train all models, record accuracy and inference time, then choose the best trade-off -> Option D

4. Quick Check:

   Balance accuracy and speed for best model [OK]

Hint: Measure both accuracy and speed, then compare [OK]

Common Mistakes:

• Ignoring speed when accuracy matters
• Choosing fastest training but poor accuracy
• Selecting model by size alone