Model evaluation best practices in Computer Vision - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Overfitting in Model Evaluation

Which of the following best describes the problem of overfitting during model evaluation?

A The model performs poorly on both training and test data.

B The model performs well on test data but poorly on training data.

C The model performs well on training data but poorly on new, unseen data.

D The model performs equally well on training and test data.

💡 Hint

Think about when a model memorizes training data but fails to generalize.

Overfitting means the model learns the training data too well, including noise, so it fails on new data.

❓ Metrics

intermediate

Choosing the Right Metric for Imbalanced Data

You have a binary classification model for detecting rare diseases in images. Which metric is most appropriate to evaluate the model?

A Accuracy

B F1 Score

C Precision

D Recall

💡 Hint

Consider a metric that balances false positives and false negatives.

F1 Score balances precision and recall, making it suitable for imbalanced classes.

❓ Predict Output

advanced

Output of Cross-Validation Accuracy Calculation

What is the output of the following Python code snippet?

Computer Vision

from sklearn.datasets import load_digits
from sklearn.model_selection import cross_val_score
from sklearn.svm import SVC

data = load_digits()
X, y = data.data, data.target
model = SVC(kernel='linear', random_state=42)
scores = cross_val_score(model, X, y, cv=5)
print(round(scores.mean(), 2))

A 0.95

B 0.85

C 0.75

D 0.65

💡 Hint

The digits dataset is relatively easy to classify with a linear SVM.

The linear SVM on the digits dataset typically achieves around 95% accuracy with 5-fold cross-validation.

🔧 Debug

advanced

Identifying the Bug in Model Evaluation Code

What error will this code raise when evaluating a classification model?

Computer Vision

from sklearn.metrics import accuracy_score

y_true = [0, 1, 2, 2, 1]
y_pred = [0, 2, 1, 2, 0]

score = accuracy_score(y_true, y_pred, average='macro')
print(score)

A TypeError: accuracy_score() got an unexpected keyword argument 'average'

B ValueError: Found input variables with inconsistent numbers of samples

C 0.4

D NameError: name 'average' is not defined

💡 Hint

Check the parameters accepted by accuracy_score.

accuracy_score does not accept an 'average' parameter; it causes a TypeError.

❓ Model Choice

expert

Best Model Choice for Multi-Class Image Classification with Limited Data

You want to build a model to classify images into 10 categories. You have only 500 labeled images. Which approach is best for evaluation and model choice?

A Train a deep CNN from scratch and evaluate on a single 80/20 train-test split.

B Use k-means clustering to label images and evaluate with silhouette score.

C Train a simple logistic regression on raw pixels and evaluate using accuracy on training data.

D Use transfer learning with a pre-trained CNN and evaluate using stratified 5-fold cross-validation.

💡 Hint

Consider data size and evaluation robustness.

Transfer learning leverages pre-trained features, and stratified cross-validation ensures balanced evaluation on limited data.

Practice

1. Why is it important to use a separate test set when evaluating a computer vision model?

easy

A. To check how well the model performs on new, unseen data

B. To make the training process faster

C. To increase the size of the training data

D. To reduce the number of model parameters

Solution

1. Step 1: Understand the purpose of a test set

   The test set is data the model has never seen before, used to check real-world performance.

2. Step 2: Compare test set role with other options

   Options B, C, and D do not relate to evaluation but to training or model design.

3. Final Answer:

   To check how well the model performs on new, unseen data -> Option A

4. Quick Check:

   Test set = unseen data check [OK]

Hint: Test set = new data to check model accuracy [OK]

Common Mistakes:

• Confusing test set with training set
• Thinking test set speeds up training
• Believing test set changes model size

2. Which of the following is the correct way to split data for model evaluation in Python using scikit-learn?

easy

A. split_train_test(data, 0.2)

B. train_test_split(data, test_size=0.2, random_state=42)

C. train_test(data, 0.2)

D. test_train_split(data, 0.2)

Solution

1. Step 1: Recall the correct function name in scikit-learn

   The function to split data is called train_test_split with parameters like test_size and random_state.

2. Step 2: Check the options for correct syntax

   Only train_test_split(data, test_size=0.2, random_state=42) uses the correct function name and parameters; others are invalid or do not exist.

3. Final Answer:

   train_test_split(data, test_size=0.2, random_state=42) -> Option B

4. Quick Check:

   Correct function = train_test_split [OK]

Hint: Remember scikit-learn function: train_test_split [OK]

Common Mistakes:

• Using wrong function names
• Missing required parameters
• Confusing order of train and test

3. Given the following code snippet, what will be the printed accuracy?

from sklearn.metrics import accuracy_score
true_labels = [1, 0, 1, 1, 0]
pred_labels = [1, 0, 0, 1, 0]
accuracy = accuracy_score(true_labels, pred_labels)
print("{:.2f}".format(round(accuracy, 2)))

medium

A. 0.60

B. 0.40

C. 0.80

D. 1.00

Solution

1. Step 1: Compare true and predicted labels

   True: [1, 0, 1, 1, 0], Predicted: [1, 0, 0, 1, 0]. Matches at positions 0,1,3,4 (4 correct out of 5).

2. Step 2: Calculate accuracy

   Accuracy = correct predictions / total = 4/5 = 0.8. Rounded to 2 decimals is 0.80.

3. Final Answer:

   0.80 -> Option C

4. Quick Check:

   Accuracy = 4/5 = 0.80 [OK]

Hint: Count matches, divide by total labels [OK]

Common Mistakes:

• Counting wrong matches
• Not rounding accuracy
• Confusing accuracy with precision

4. You trained a model but the test accuracy is much higher than the training accuracy. What is the most likely issue?

medium

A. Data leakage between training and test sets

B. Model is underfitting the training data

C. Test set is too small

D. Training data is too large

Solution

1. Step 1: Understand unusual accuracy pattern

   Test accuracy higher than training is unusual and often means test data was seen during training.

2. Step 2: Identify cause from options

   Data leakage means test data accidentally used in training, causing inflated test accuracy.

3. Final Answer:

   Data leakage between training and test sets -> Option A

4. Quick Check:

   High test accuracy > training = data leakage [OK]

Hint: High test accuracy than train? Check data leakage [OK]

Common Mistakes:

• Assuming underfitting causes higher test accuracy
• Ignoring data leakage possibility
• Blaming test set size without evidence

5. You want to evaluate a computer vision model for detecting rare objects in images. Which evaluation metric is best to use and why?

hard

A. Confusion matrix, because it shows training time

B. Accuracy, because it shows overall correct predictions

C. Mean Squared Error, because it measures prediction error

D. F1 score, because it balances precision and recall for imbalanced data

Solution

1. Step 1: Understand the problem of rare object detection

   Rare objects mean data is imbalanced; many negatives, few positives.

2. Step 2: Choose metric suitable for imbalanced data

   F1 score balances precision (correct positive predictions) and recall (finding all positives), ideal for rare classes.

3. Final Answer:

   F1 score, because it balances precision and recall for imbalanced data -> Option D

4. Quick Check:

   Rare class? Use F1 score [OK]

Hint: Rare classes? Use F1 score for balance [OK]

Common Mistakes:

• Using accuracy which hides imbalance
• Confusing regression metrics with classification
• Misunderstanding confusion matrix purpose