Top-K accuracy in Computer Vision - Model Pipeline Trace

Model Pipeline - Top-K accuracy

This pipeline shows how a computer vision model learns to recognize images and how Top-K accuracy measures if the correct label is among the model's top K guesses.

Data Flow - 6 Stages

1Data in

1000 images x 64 x 64 x 3→Raw images loaded with labels→1000 images x 64 x 64 x 3

Image of a cat with label 'cat'

↓

2Preprocessing

1000 images x 64 x 64 x 3→Normalize pixel values to 0-1 range→1000 images x 64 x 64 x 3

Pixel values scaled from 0-255 to 0-1

↓

3Feature Engineering

1000 images x 64 x 64 x 3→Extract features using convolutional layers→1000 images x 16 x 16 x 32

Feature maps highlighting edges and textures

↓

4Model Trains

1000 images x 16 x 16 x 32→Train CNN classifier to predict 10 classes→1000 predictions x 10 classes

Prediction vector like [0.1, 0.05, 0.6, ..., 0.02]

↓

5Metrics Improve

1000 predictions x 10 classes→Calculate Top-1 and Top-3 accuracy→Accuracy scores

Top-1 accuracy = 0.75, Top-3 accuracy = 0.92

↓

6Prediction

1 image x 64 x 64 x 3→Model outputs probabilities for each class→1 prediction x 10 classes

Output vector [0.05, 0.1, 0.7, ..., 0.01]

Training Trace - Epoch by Epoch

Loss
2.0 |****
1.5 |*** 
1.0 |**  
0.5 |*   
0.0 +----
      1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.8	0.35	Model starts learning, accuracy low
2	1.2	0.55	Loss decreases, accuracy improves
3	0.9	0.68	Model getting better at classification
4	0.7	0.75	Accuracy approaching good performance
5	0.6	0.80	Model converging with good accuracy

Prediction Trace - 5 Layers

Layer 1: Input Image

Layer 2: Convolutional Layers

Layer 3: Fully Connected Layer

Layer 4: Softmax Activation

Layer 5: Top-K Accuracy Check

Model Quiz - 3 Questions

What does Top-3 accuracy measure in this model?

A If the correct label is among the top 3 predicted classes

B If the model predicts exactly 3 classes correctly

C If the model's top prediction is correct

D If the model predicts any class with probability above 0.3

Key Insight

Top-K accuracy helps us understand if the model's correct answer is among its best guesses, which is useful when multiple classes look similar. Watching loss decrease and accuracy increase shows the model is learning well.

Practice

1. What does Top-K accuracy measure in a classification model?

easy

A. If the true label is among the top K predicted labels

B. The accuracy of the model's single best prediction only

C. The time taken to make K predictions

D. The number of classes in the dataset

Solution

1. Step 1: Understand the definition of Top-K accuracy

   Top-K accuracy checks if the correct label is within the top K guesses made by the model, not just the top 1.

2. Step 2: Compare options with the definition

   Only If the true label is among the top K predicted labels correctly states that Top-K accuracy measures if the true label is in the top K predictions.

3. Final Answer:

   If the true label is among the top K predicted labels -> Option A

4. Quick Check:

   Top-K accuracy = True label in top K predictions [OK]

Hint: Top-K means checking top K guesses, not just one [OK]

Common Mistakes:

• Confusing Top-K accuracy with single-label accuracy
• Thinking it measures prediction speed
• Assuming it counts total classes

2. Which of the following is the correct way to compute Top-3 accuracy using PyTorch's topk method on model outputs outputs and true labels labels?

easy

A. pred = outputs.max(3); correct = pred.eq(labels).sum().item()

B. correct = outputs.topk(3).eq(labels).sum().item()

C. _, pred = outputs.topk(3, dim=1); correct = pred.eq(labels.view(-1,1)).sum().item()

D. _, pred = outputs.topk(1, dim=0); correct = pred.eq(labels).sum().item()

Solution

1. Step 1: Understand PyTorch topk usage

   The topk(k, dim=1) returns top k values and indices along dimension 1 (classes). We want indices for predictions.

2. Step 2: Match predictions with labels

   Reshape labels to (-1,1) to compare with top-k predictions and count matches with eq and sum.

3. Final Answer:

   _, pred = outputs.topk(3, dim=1); correct = pred.eq(labels.view(-1,1)).sum().item() -> Option C

4. Quick Check:

   Use topk with dim=1 and compare with labels reshaped [OK]

Hint: Use topk(dim=1) and reshape labels for comparison [OK]

Common Mistakes:

• Using max instead of topk for multiple predictions
• Wrong dimension in topk call
• Not reshaping labels for comparison

3. Given the following PyTorch code snippet, what is the printed Top-2 accuracy count?

import torch
outputs = torch.tensor([[0.1, 0.8, 0.05, 0.05],
                        [0.4, 0.3, 0.2, 0.1],
                        [0.25, 0.25, 0.25, 0.25]])
labels = torch.tensor([1, 2, 3])
_, pred = outputs.topk(2, dim=1)
correct = pred.eq(labels.view(-1,1)).sum().item()
print(correct)

medium

A. 2

B. 1

C. 3

D. 0

Solution

1. Step 1: Identify top 2 predictions per sample

   For each row: - Row 1: top 2 indices are [1, 0] (0.8, 0.1) - Row 2: top 2 indices are [0, 1] (0.4, 0.3) - Row 3: top 2 indices are [0, 1] (both 0.25, tie broken by index)

2. Step 2: Check if true label is in top 2 predictions

   Labels are [1, 2, 3]: - Sample 1: label 1 in [1,0] -> yes - Sample 2: label 2 in [0,1] -> no - Sample 3: label 3 in [0,1] -> no

3. Final Answer:

   1 -> Option B

4. Quick Check:

   Only one label in top 2 predictions [OK]

Hint: Check top K indices and compare with labels one by one [OK]

Common Mistakes:

• Assuming all labels are in top 2
• Ignoring tie-breaking in topk
• Not reshaping labels for comparison

4. You wrote this code to compute Top-5 accuracy but it always returns zero. What is the bug?

_, pred = outputs.topk(5)
correct = pred.eq(labels).sum().item()

medium

A. Missing dimension argument in topk causes wrong axis selection

B. Labels should be converted to float before comparison

C. topk should be called with k=1 for Top-5 accuracy

D. Using sum().item() returns a tensor, not a number

Solution

1. Step 1: Check topk usage without dimension

   Calling topk(5) without dim defaults to dim=0, which is incorrect for class predictions along dim=1.

2. Step 2: Understand effect on predictions and comparison

   Wrong dimension means predicted indices do not align with labels, so pred.eq(labels) never matches, resulting in zero correct.

3. Final Answer:

   Missing dimension argument in topk causes wrong axis selection -> Option A

4. Quick Check:

   Always specify dim=1 for class predictions in topk [OK]

Hint: Always specify dim=1 in topk for class dimension [OK]

Common Mistakes:

• Forgetting dim argument in topk
• Converting labels unnecessarily
• Misunderstanding sum().item() output

5. You have a model with 100 classes and want to report Top-1 and Top-5 accuracy on a test set. Which approach best handles the evaluation efficiently and correctly?

hard

A. Use topk(1, dim=1) for Top-5 accuracy and topk(5, dim=1) for Top-1 accuracy

B. Compute Top-1 accuracy by max(dim=0) and Top-5 by topk(5, dim=0) without reshaping labels

C. Calculate Top-5 accuracy by checking if label is in top 5 predictions using a for loop over each sample

D. Use topk(5, dim=1) on model outputs, compare with labels reshaped, then compute Top-1 by checking if label equals top prediction

Solution

1. Step 1: Understand correct usage of topk for Top-K accuracy

   Top-5 accuracy requires topk(5, dim=1) to get top 5 class indices per sample. Labels must be reshaped to compare with these indices.

2. Step 2: Compute Top-1 accuracy separately

   Top-1 accuracy is checking if label equals the top prediction (max or topk with k=1). This is done by comparing labels with top prediction indices.

3. Final Answer:

   Use topk(5, dim=1) on model outputs, compare with labels reshaped, then compute Top-1 by checking if label equals top prediction -> Option D

4. Quick Check:

   Top-K needs topk(dim=1) and label reshape; Top-1 is top prediction check [OK]

Hint: Top-K needs topk(dim=1) and label reshape; Top-1 is top prediction check [OK]

Common Mistakes:

• Using wrong dimension in topk
• Mixing up Top-1 and Top-5 calls
• Not reshaping labels for comparison
• Using loops instead of vectorized operations