Top-K accuracy in Computer Vision - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is Top-K accuracy in machine learning?

Top-K accuracy measures if the correct answer is among the model's top K guesses. For example, Top-3 accuracy checks if the true label is in the top 3 predicted labels.

beginner

Why is Top-K accuracy useful in computer vision tasks?

Because some images can be ambiguous or have multiple possible labels, Top-K accuracy gives a more forgiving measure by checking if the correct label is within the top guesses, not just the top one.

intermediate

How do you calculate Top-5 accuracy for a classification model?

For each image, check if the true label is in the model's 5 highest probability predictions. The Top-5 accuracy is the percentage of images where this is true.

intermediate

Code snippet: What does this Python code do? preds = [[0.1, 0.7, 0.2], [0.3, 0.4, 0.3]] true_labels = [1, 0] # Calculate Top-2 accuracy

This code checks if the true label is in the top 2 predicted probabilities for each example. It counts how many times this is true and divides by total examples to get Top-2 accuracy.

beginner

What is the difference between Top-1 accuracy and Top-K accuracy?

Top-1 accuracy checks if the model's single highest prediction matches the true label. Top-K accuracy checks if the true label is within the top K predictions, allowing more chances to be correct.

What does Top-3 accuracy measure?

A If the model predicts exactly 3 labels

B If the true label is the single highest prediction

C If the model's accuracy is above 3%

D If the true label is among the top 3 predicted labels

Top-3 accuracy checks if the correct label is within the model's top 3 guesses.

Why might Top-K accuracy be preferred over Top-1 accuracy?

A It is easier to calculate

B It always gives higher accuracy values

C It allows for some prediction uncertainty by considering multiple guesses

D It ignores the true label

Top-K accuracy is useful when multiple guesses are acceptable, reflecting model performance better in ambiguous cases.

In Top-5 accuracy, what does the number 5 represent?

A Number of top predictions checked for correctness

B Number of classes in the dataset

C Number of training epochs

D Number of samples tested

The 5 means the model's top 5 predicted labels are checked to see if the true label is among them.

If a model has 90% Top-1 accuracy and 98% Top-5 accuracy, what does this mean?

A The model is better at predicting exactly one label

B The model's true label is often in its top 5 guesses but less often the top guess

C The model is overfitting

D The model's accuracy is decreasing

Higher Top-5 accuracy means the true label is usually within the top 5 predictions, even if not the top one.

Which scenario best suits using Top-K accuracy?

A When multiple answers could be acceptable or similar

B When only one correct answer exists and must be exact

C When the dataset has only two classes

D When the model is unsupervised

Top-K accuracy is useful when multiple possible answers are acceptable or close in meaning.

Explain what Top-K accuracy is and why it is important in evaluating classification models.

Think about how checking multiple top guesses helps understand model performance better.

Describe how you would calculate Top-3 accuracy for a model's predictions on a test set.

Focus on comparing true labels with the model's top 3 predicted labels.

Practice

1. What does Top-K accuracy measure in a classification model?

easy

A. If the true label is among the top K predicted labels

B. The accuracy of the model's single best prediction only

C. The time taken to make K predictions

D. The number of classes in the dataset

Solution

1. Step 1: Understand the definition of Top-K accuracy

   Top-K accuracy checks if the correct label is within the top K guesses made by the model, not just the top 1.

2. Step 2: Compare options with the definition

   Only If the true label is among the top K predicted labels correctly states that Top-K accuracy measures if the true label is in the top K predictions.

3. Final Answer:

   If the true label is among the top K predicted labels -> Option A

4. Quick Check:

   Top-K accuracy = True label in top K predictions [OK]

Hint: Top-K means checking top K guesses, not just one [OK]

Common Mistakes:

• Confusing Top-K accuracy with single-label accuracy
• Thinking it measures prediction speed
• Assuming it counts total classes

2. Which of the following is the correct way to compute Top-3 accuracy using PyTorch's topk method on model outputs outputs and true labels labels?

easy

A. pred = outputs.max(3); correct = pred.eq(labels).sum().item()

B. correct = outputs.topk(3).eq(labels).sum().item()

C. _, pred = outputs.topk(3, dim=1); correct = pred.eq(labels.view(-1,1)).sum().item()

D. _, pred = outputs.topk(1, dim=0); correct = pred.eq(labels).sum().item()

Solution

1. Step 1: Understand PyTorch topk usage

   The topk(k, dim=1) returns top k values and indices along dimension 1 (classes). We want indices for predictions.

2. Step 2: Match predictions with labels

   Reshape labels to (-1,1) to compare with top-k predictions and count matches with eq and sum.

3. Final Answer:

   _, pred = outputs.topk(3, dim=1); correct = pred.eq(labels.view(-1,1)).sum().item() -> Option C

4. Quick Check:

   Use topk with dim=1 and compare with labels reshaped [OK]

Hint: Use topk(dim=1) and reshape labels for comparison [OK]

Common Mistakes:

• Using max instead of topk for multiple predictions
• Wrong dimension in topk call
• Not reshaping labels for comparison

3. Given the following PyTorch code snippet, what is the printed Top-2 accuracy count?

import torch
outputs = torch.tensor([[0.1, 0.8, 0.05, 0.05],
                        [0.4, 0.3, 0.2, 0.1],
                        [0.25, 0.25, 0.25, 0.25]])
labels = torch.tensor([1, 2, 3])
_, pred = outputs.topk(2, dim=1)
correct = pred.eq(labels.view(-1,1)).sum().item()
print(correct)

medium

A. 2

B. 1

C. 3

D. 0

Solution

1. Step 1: Identify top 2 predictions per sample

   For each row: - Row 1: top 2 indices are [1, 0] (0.8, 0.1) - Row 2: top 2 indices are [0, 1] (0.4, 0.3) - Row 3: top 2 indices are [0, 1] (both 0.25, tie broken by index)

2. Step 2: Check if true label is in top 2 predictions

   Labels are [1, 2, 3]: - Sample 1: label 1 in [1,0] -> yes - Sample 2: label 2 in [0,1] -> no - Sample 3: label 3 in [0,1] -> no

3. Final Answer:

   1 -> Option B

4. Quick Check:

   Only one label in top 2 predictions [OK]

Hint: Check top K indices and compare with labels one by one [OK]

Common Mistakes:

• Assuming all labels are in top 2
• Ignoring tie-breaking in topk
• Not reshaping labels for comparison

4. You wrote this code to compute Top-5 accuracy but it always returns zero. What is the bug?

_, pred = outputs.topk(5)
correct = pred.eq(labels).sum().item()

medium

A. Missing dimension argument in topk causes wrong axis selection

B. Labels should be converted to float before comparison

C. topk should be called with k=1 for Top-5 accuracy

D. Using sum().item() returns a tensor, not a number

Solution

1. Step 1: Check topk usage without dimension

   Calling topk(5) without dim defaults to dim=0, which is incorrect for class predictions along dim=1.

2. Step 2: Understand effect on predictions and comparison

   Wrong dimension means predicted indices do not align with labels, so pred.eq(labels) never matches, resulting in zero correct.

3. Final Answer:

   Missing dimension argument in topk causes wrong axis selection -> Option A

4. Quick Check:

   Always specify dim=1 for class predictions in topk [OK]

Hint: Always specify dim=1 in topk for class dimension [OK]

Common Mistakes:

• Forgetting dim argument in topk
• Converting labels unnecessarily
• Misunderstanding sum().item() output

5. You have a model with 100 classes and want to report Top-1 and Top-5 accuracy on a test set. Which approach best handles the evaluation efficiently and correctly?

hard

A. Use topk(1, dim=1) for Top-5 accuracy and topk(5, dim=1) for Top-1 accuracy

B. Compute Top-1 accuracy by max(dim=0) and Top-5 by topk(5, dim=0) without reshaping labels

C. Calculate Top-5 accuracy by checking if label is in top 5 predictions using a for loop over each sample

D. Use topk(5, dim=1) on model outputs, compare with labels reshaped, then compute Top-1 by checking if label equals top prediction

Solution

1. Step 1: Understand correct usage of topk for Top-K accuracy

   Top-5 accuracy requires topk(5, dim=1) to get top 5 class indices per sample. Labels must be reshaped to compare with these indices.

2. Step 2: Compute Top-1 accuracy separately

   Top-1 accuracy is checking if label equals the top prediction (max or topk with k=1). This is done by comparing labels with top prediction indices.

3. Final Answer:

   Use topk(5, dim=1) on model outputs, compare with labels reshaped, then compute Top-1 by checking if label equals top prediction -> Option D

4. Quick Check:

   Top-K needs topk(dim=1) and label reshape; Top-1 is top prediction check [OK]

Hint: Top-K needs topk(dim=1) and label reshape; Top-1 is top prediction check [OK]

Common Mistakes:

• Using wrong dimension in topk
• Mixing up Top-1 and Top-5 calls
• Not reshaping labels for comparison
• Using loops instead of vectorized operations