Top-K accuracy in Computer Vision

Introduction

Top-K accuracy helps us check if the right answer is among the top K guesses of a model, not just the very top guess.

When you want to see if the correct label is within the top 3 or 5 guesses of an image classifier.

When exact single prediction is hard, but a few good guesses are useful, like in photo tagging apps.

When evaluating models on tasks with many classes, like recognizing many types of animals.

When you want to compare models by how often they include the right answer in their top choices.

Syntax

Computer Vision

top_k_accuracy_score(y_true, y_pred_probs, k=5)

# y_true: true labels
# y_pred_probs: predicted probabilities for each class
# k: number of top guesses to consider

The function checks if the true label is in the top K predicted classes.

Higher K means easier to get correct, but less strict accuracy.

Examples

Checks if the top guess matches the true label exactly.

Computer Vision

top_k_accuracy_score([2, 0, 1], [[0.1, 0.2, 0.7], [0.8, 0.1, 0.1], [0.2, 0.6, 0.2]], k=1)

Checks if the true label is in the top 2 guesses.

Computer Vision

top_k_accuracy_score([2, 0, 1], [[0.1, 0.2, 0.7], [0.8, 0.1, 0.1], [0.2, 0.6, 0.2]], k=2)

Sample Model

This program shows how often the model's top guess is correct (top-1) and how often the correct label is in the top 2 guesses (top-2).

Computer Vision

from sklearn.metrics import top_k_accuracy_score

# True labels for 4 images
y_true = [0, 1, 2, 3]

# Predicted probabilities for 4 classes
# Each inner list sums to 1 and shows model confidence
y_pred_probs = [
    [0.7, 0.1, 0.1, 0.1],  # Correct class 0 is top guess
    [0.2, 0.3, 0.4, 0.1],  # Correct class 1 is second guess
    [0.1, 0.2, 0.5, 0.2],  # Correct class 2 is top guess
    [0.1, 0.1, 0.2, 0.6]   # Correct class 3 is top guess
]

# Calculate top-1 accuracy (exact match)
top1 = top_k_accuracy_score(y_true, y_pred_probs, k=1)

# Calculate top-2 accuracy (correct label in top 2 guesses)
top2 = top_k_accuracy_score(y_true, y_pred_probs, k=2)

print(f"Top-1 accuracy: {top1:.2f}")
print(f"Top-2 accuracy: {top2:.2f}")

Important Notes

Top-K accuracy is useful when multiple guesses are acceptable.

Choosing K depends on your task and how many guesses you want to consider.

Top-K accuracy is common in image recognition challenges with many classes.

Summary

Top-K accuracy checks if the true label is within the top K model guesses.

It helps measure model performance beyond just the single best guess.

Use it when multiple possible answers are useful or when classes are many.

Practice

1. What does Top-K accuracy measure in a classification model?

easy

A. If the true label is among the top K predicted labels

B. The accuracy of the model's single best prediction only

C. The time taken to make K predictions

D. The number of classes in the dataset

Solution

1. Step 1: Understand the definition of Top-K accuracy

   Top-K accuracy checks if the correct label is within the top K guesses made by the model, not just the top 1.

2. Step 2: Compare options with the definition

   Only If the true label is among the top K predicted labels correctly states that Top-K accuracy measures if the true label is in the top K predictions.

3. Final Answer:

   If the true label is among the top K predicted labels -> Option A

4. Quick Check:

   Top-K accuracy = True label in top K predictions [OK]

Hint: Top-K means checking top K guesses, not just one [OK]

Common Mistakes:

• Confusing Top-K accuracy with single-label accuracy
• Thinking it measures prediction speed
• Assuming it counts total classes

2. Which of the following is the correct way to compute Top-3 accuracy using PyTorch's topk method on model outputs outputs and true labels labels?

easy

A. pred = outputs.max(3); correct = pred.eq(labels).sum().item()

B. correct = outputs.topk(3).eq(labels).sum().item()

C. _, pred = outputs.topk(3, dim=1); correct = pred.eq(labels.view(-1,1)).sum().item()

D. _, pred = outputs.topk(1, dim=0); correct = pred.eq(labels).sum().item()

Solution

1. Step 1: Understand PyTorch topk usage

   The topk(k, dim=1) returns top k values and indices along dimension 1 (classes). We want indices for predictions.

2. Step 2: Match predictions with labels

   Reshape labels to (-1,1) to compare with top-k predictions and count matches with eq and sum.

3. Final Answer:

   _, pred = outputs.topk(3, dim=1); correct = pred.eq(labels.view(-1,1)).sum().item() -> Option C

4. Quick Check:

   Use topk with dim=1 and compare with labels reshaped [OK]

Hint: Use topk(dim=1) and reshape labels for comparison [OK]

Common Mistakes:

• Using max instead of topk for multiple predictions
• Wrong dimension in topk call
• Not reshaping labels for comparison

3. Given the following PyTorch code snippet, what is the printed Top-2 accuracy count?

import torch
outputs = torch.tensor([[0.1, 0.8, 0.05, 0.05],
                        [0.4, 0.3, 0.2, 0.1],
                        [0.25, 0.25, 0.25, 0.25]])
labels = torch.tensor([1, 2, 3])
_, pred = outputs.topk(2, dim=1)
correct = pred.eq(labels.view(-1,1)).sum().item()
print(correct)

medium

A. 2

B. 1

C. 3

D. 0

Solution

1. Step 1: Identify top 2 predictions per sample

   For each row: - Row 1: top 2 indices are [1, 0] (0.8, 0.1) - Row 2: top 2 indices are [0, 1] (0.4, 0.3) - Row 3: top 2 indices are [0, 1] (both 0.25, tie broken by index)

2. Step 2: Check if true label is in top 2 predictions

   Labels are [1, 2, 3]: - Sample 1: label 1 in [1,0] -> yes - Sample 2: label 2 in [0,1] -> no - Sample 3: label 3 in [0,1] -> no

3. Final Answer:

   1 -> Option B

4. Quick Check:

   Only one label in top 2 predictions [OK]

Hint: Check top K indices and compare with labels one by one [OK]

Common Mistakes:

• Assuming all labels are in top 2
• Ignoring tie-breaking in topk
• Not reshaping labels for comparison

4. You wrote this code to compute Top-5 accuracy but it always returns zero. What is the bug?

_, pred = outputs.topk(5)
correct = pred.eq(labels).sum().item()

medium

A. Missing dimension argument in topk causes wrong axis selection

B. Labels should be converted to float before comparison

C. topk should be called with k=1 for Top-5 accuracy

D. Using sum().item() returns a tensor, not a number

Solution

1. Step 1: Check topk usage without dimension

   Calling topk(5) without dim defaults to dim=0, which is incorrect for class predictions along dim=1.

2. Step 2: Understand effect on predictions and comparison

   Wrong dimension means predicted indices do not align with labels, so pred.eq(labels) never matches, resulting in zero correct.

3. Final Answer:

   Missing dimension argument in topk causes wrong axis selection -> Option A

4. Quick Check:

   Always specify dim=1 for class predictions in topk [OK]

Hint: Always specify dim=1 in topk for class dimension [OK]

Common Mistakes:

• Forgetting dim argument in topk
• Converting labels unnecessarily
• Misunderstanding sum().item() output

5. You have a model with 100 classes and want to report Top-1 and Top-5 accuracy on a test set. Which approach best handles the evaluation efficiently and correctly?

hard

A. Use topk(1, dim=1) for Top-5 accuracy and topk(5, dim=1) for Top-1 accuracy

B. Compute Top-1 accuracy by max(dim=0) and Top-5 by topk(5, dim=0) without reshaping labels

C. Calculate Top-5 accuracy by checking if label is in top 5 predictions using a for loop over each sample

D. Use topk(5, dim=1) on model outputs, compare with labels reshaped, then compute Top-1 by checking if label equals top prediction

Solution

1. Step 1: Understand correct usage of topk for Top-K accuracy

   Top-5 accuracy requires topk(5, dim=1) to get top 5 class indices per sample. Labels must be reshaped to compare with these indices.

2. Step 2: Compute Top-1 accuracy separately

   Top-1 accuracy is checking if label equals the top prediction (max or topk with k=1). This is done by comparing labels with top prediction indices.

3. Final Answer:

   Use topk(5, dim=1) on model outputs, compare with labels reshaped, then compute Top-1 by checking if label equals top prediction -> Option D

4. Quick Check:

   Top-K needs topk(dim=1) and label reshape; Top-1 is top prediction check [OK]

Hint: Top-K needs topk(dim=1) and label reshape; Top-1 is top prediction check [OK]

Common Mistakes:

• Using wrong dimension in topk
• Mixing up Top-1 and Top-5 calls
• Not reshaping labels for comparison
• Using loops instead of vectorized operations