Why Top-K accuracy in Computer Vision? - Purpose & Use Cases

The Big Idea

What if your model's second or third guess is right most of the time, but you never knew it?

The Scenario

Imagine you are sorting photos of animals by guessing what animal is in each picture. You write down only your first guess for each photo and check if it is correct.

The Problem

This way is frustrating because sometimes your first guess is wrong, but your second or third guess is right. You miss those almost-correct answers and think your guesses are worse than they really are.

The Solution

Top-K accuracy lets you check if the correct answer is anywhere in your top K guesses, not just the first one. This gives a fairer score and shows how well your model is really doing.

Before vs After

✗ Before

correct = (prediction == true_label)
accuracy = sum(correct) / len(correct)

✓ After

correct = [true_label in preds for preds in top_k_predictions]
top_k_accuracy = sum(correct) / len(correct)

What It Enables

Top-K accuracy helps us understand how often the right answer is close to the top predictions, making model evaluation more realistic and useful.

Real Life Example

In a photo app that suggests animal names, Top-K accuracy shows if the right animal is among the top 3 suggestions, even if it's not the first guess, improving user experience.

Key Takeaways

Manual single-guess checks miss near-correct answers.

Top-K accuracy checks if the correct answer is in the top K guesses.

This gives a better picture of model performance and usefulness.

Practice

1. What does Top-K accuracy measure in a classification model?

easy

A. If the true label is among the top K predicted labels

B. The accuracy of the model's single best prediction only

C. The time taken to make K predictions

D. The number of classes in the dataset

Solution

1. Step 1: Understand the definition of Top-K accuracy

   Top-K accuracy checks if the correct label is within the top K guesses made by the model, not just the top 1.

2. Step 2: Compare options with the definition

   Only If the true label is among the top K predicted labels correctly states that Top-K accuracy measures if the true label is in the top K predictions.

3. Final Answer:

   If the true label is among the top K predicted labels -> Option A

4. Quick Check:

   Top-K accuracy = True label in top K predictions [OK]

Hint: Top-K means checking top K guesses, not just one [OK]

Common Mistakes:

• Confusing Top-K accuracy with single-label accuracy
• Thinking it measures prediction speed
• Assuming it counts total classes

2. Which of the following is the correct way to compute Top-3 accuracy using PyTorch's topk method on model outputs outputs and true labels labels?

easy

A. pred = outputs.max(3); correct = pred.eq(labels).sum().item()

B. correct = outputs.topk(3).eq(labels).sum().item()

C. _, pred = outputs.topk(3, dim=1); correct = pred.eq(labels.view(-1,1)).sum().item()

D. _, pred = outputs.topk(1, dim=0); correct = pred.eq(labels).sum().item()

Solution

1. Step 1: Understand PyTorch topk usage

   The topk(k, dim=1) returns top k values and indices along dimension 1 (classes). We want indices for predictions.

2. Step 2: Match predictions with labels

   Reshape labels to (-1,1) to compare with top-k predictions and count matches with eq and sum.

3. Final Answer:

   _, pred = outputs.topk(3, dim=1); correct = pred.eq(labels.view(-1,1)).sum().item() -> Option C

4. Quick Check:

   Use topk with dim=1 and compare with labels reshaped [OK]

Hint: Use topk(dim=1) and reshape labels for comparison [OK]

Common Mistakes:

• Using max instead of topk for multiple predictions
• Wrong dimension in topk call
• Not reshaping labels for comparison

3. Given the following PyTorch code snippet, what is the printed Top-2 accuracy count?

import torch
outputs = torch.tensor([[0.1, 0.8, 0.05, 0.05],
                        [0.4, 0.3, 0.2, 0.1],
                        [0.25, 0.25, 0.25, 0.25]])
labels = torch.tensor([1, 2, 3])
_, pred = outputs.topk(2, dim=1)
correct = pred.eq(labels.view(-1,1)).sum().item()
print(correct)

medium

A. 2

B. 1

C. 3

D. 0

Solution

1. Step 1: Identify top 2 predictions per sample

   For each row: - Row 1: top 2 indices are [1, 0] (0.8, 0.1) - Row 2: top 2 indices are [0, 1] (0.4, 0.3) - Row 3: top 2 indices are [0, 1] (both 0.25, tie broken by index)

2. Step 2: Check if true label is in top 2 predictions

   Labels are [1, 2, 3]: - Sample 1: label 1 in [1,0] -> yes - Sample 2: label 2 in [0,1] -> no - Sample 3: label 3 in [0,1] -> no

3. Final Answer:

   1 -> Option B

4. Quick Check:

   Only one label in top 2 predictions [OK]

Hint: Check top K indices and compare with labels one by one [OK]

Common Mistakes:

• Assuming all labels are in top 2
• Ignoring tie-breaking in topk
• Not reshaping labels for comparison

4. You wrote this code to compute Top-5 accuracy but it always returns zero. What is the bug?

_, pred = outputs.topk(5)
correct = pred.eq(labels).sum().item()

medium

A. Missing dimension argument in topk causes wrong axis selection

B. Labels should be converted to float before comparison

C. topk should be called with k=1 for Top-5 accuracy

D. Using sum().item() returns a tensor, not a number

Solution

1. Step 1: Check topk usage without dimension

   Calling topk(5) without dim defaults to dim=0, which is incorrect for class predictions along dim=1.

2. Step 2: Understand effect on predictions and comparison

   Wrong dimension means predicted indices do not align with labels, so pred.eq(labels) never matches, resulting in zero correct.

3. Final Answer:

   Missing dimension argument in topk causes wrong axis selection -> Option A

4. Quick Check:

   Always specify dim=1 for class predictions in topk [OK]

Hint: Always specify dim=1 in topk for class dimension [OK]

Common Mistakes:

• Forgetting dim argument in topk
• Converting labels unnecessarily
• Misunderstanding sum().item() output

5. You have a model with 100 classes and want to report Top-1 and Top-5 accuracy on a test set. Which approach best handles the evaluation efficiently and correctly?

hard

A. Use topk(1, dim=1) for Top-5 accuracy and topk(5, dim=1) for Top-1 accuracy

B. Compute Top-1 accuracy by max(dim=0) and Top-5 by topk(5, dim=0) without reshaping labels

C. Calculate Top-5 accuracy by checking if label is in top 5 predictions using a for loop over each sample

D. Use topk(5, dim=1) on model outputs, compare with labels reshaped, then compute Top-1 by checking if label equals top prediction

Solution

1. Step 1: Understand correct usage of topk for Top-K accuracy

   Top-5 accuracy requires topk(5, dim=1) to get top 5 class indices per sample. Labels must be reshaped to compare with these indices.

2. Step 2: Compute Top-1 accuracy separately

   Top-1 accuracy is checking if label equals the top prediction (max or topk with k=1). This is done by comparing labels with top prediction indices.

3. Final Answer:

   Use topk(5, dim=1) on model outputs, compare with labels reshaped, then compute Top-1 by checking if label equals top prediction -> Option D

4. Quick Check:

   Top-K needs topk(dim=1) and label reshape; Top-1 is top prediction check [OK]

Hint: Top-K needs topk(dim=1) and label reshape; Top-1 is top prediction check [OK]

Common Mistakes:

• Using wrong dimension in topk
• Mixing up Top-1 and Top-5 calls
• Not reshaping labels for comparison
• Using loops instead of vectorized operations