Segmentation evaluation (IoU, Dice) in Computer Vision - Model Pipeline Trace

Model Pipeline - Segmentation evaluation (IoU, Dice)

This pipeline shows how a computer learns to separate objects in images by predicting masks. It then checks how well the predicted masks match the real masks using two scores: IoU and Dice. These scores tell us how much the predicted and real areas overlap.

Data Flow - 5 Stages

1Input images and ground truth masks

100 images x 128 x 128 pixels→Load images and their true segmentation masks→100 images x 128 x 128 pixels (images), 100 masks x 128 x 128 pixels (masks)

Image of a cat and mask showing cat pixels as 1, background as 0

↓

2Preprocessing

100 images x 128 x 128 pixels→Normalize pixel values to 0-1 range→100 images x 128 x 128 pixels (normalized)

Pixel value 150 becomes 0.59

↓

3Model prediction

100 images x 128 x 128 pixels→Model predicts mask probabilities for each pixel→100 predicted masks x 128 x 128 pixels (values 0-1)

Pixel predicted as 0.8 means 80% chance of object

↓

4Thresholding

100 predicted masks x 128 x 128 pixels (0-1)→Convert probabilities to binary masks using 0.5 cutoff→100 binary predicted masks x 128 x 128 pixels (0 or 1)

Pixel 0.8 becomes 1, pixel 0.3 becomes 0

↓

5Evaluation metrics calculation

100 binary predicted masks and 100 ground truth masks x 128 x 128 pixels→Calculate IoU and Dice scores for each mask pair→100 IoU scores, 100 Dice scores (values 0-1)

IoU = 0.75, Dice = 0.85 for one image

Training Trace - Epoch by Epoch

Loss: 0.65 |****
       0.50 |******
       0.40 |********
       0.32 |**********
       0.28 |***********
Epochs -> 1    2    3    4    5

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.65	0.60	Model starts learning, loss high, accuracy low
2	0.50	0.72	Loss decreases, accuracy improves
3	0.40	0.80	Model getting better at segmentation
4	0.32	0.85	Loss continues to drop, accuracy rises
5	0.28	0.88	Training converging with good accuracy

Prediction Trace - 5 Layers

Layer 1: Input image

Layer 2: Model convolution layers

Layer 3: Final sigmoid layer

Layer 4: Thresholding

Layer 5: IoU and Dice calculation

Model Quiz - 3 Questions

What does an IoU score of 1.0 mean?

A Predicted mask is empty

B No overlap between predicted and true masks

C Perfect overlap between predicted and true masks

D Ground truth mask is empty

Key Insight

IoU and Dice scores are important to measure how well a model segments objects in images. They focus on overlap quality, which accuracy alone cannot capture. This helps us understand if the model truly learns to separate objects from background.

Practice

1. What does the Intersection over Union (IoU) metric measure in image segmentation?

easy

A. The overlap between predicted and true masks divided by their union

B. The difference between predicted and true masks

C. The total number of pixels in the predicted mask

D. The ratio of true positives to false negatives

Solution

1. Step 1: Understand IoU definition

   IoU measures how much the predicted mask and true mask overlap compared to their combined area.

2. Step 2: Apply IoU formula

   IoU = (Area of Overlap) / (Area of Union) between predicted and true masks.

3. Final Answer:

   The overlap between predicted and true masks divided by their union -> Option A

4. Quick Check:

   IoU = Overlap / Union [OK]

Hint: IoU = Overlap divided by combined area [OK]

Common Mistakes:

• Confusing IoU with difference or subtraction
• Using only predicted mask area
• Mixing IoU with accuracy or precision

2. Which of the following is the correct formula for Dice coefficient between predicted mask P and true mask T?

easy

A. Dice = |P| / |T|

B. Dice = (2 * |P ∩ T|) / (|P| + |T|)

C. Dice = |P ∪ T| / (|P| + |T|)

D. Dice = |P ∩ T| / |P ∪ T|

Solution

1. Step 1: Recall Dice coefficient formula

   Dice coefficient is defined as twice the overlap divided by the sum of sizes of predicted and true masks.

2. Step 2: Match formula to options

   Dice = (2 * |P ∩ T|) / (|P| + |T|) matches Dice = (2 * intersection) / (sum of areas).

3. Final Answer:

   Dice = (2 * |P ∩ T|) / (|P| + |T|) -> Option B

4. Quick Check:

   Dice = 2*Overlap / (Sum areas) [OK]

Hint: Dice doubles overlap, divides by sum of mask sizes [OK]

Common Mistakes:

• Confusing Dice with IoU formula
• Using union instead of sum of areas
• Forgetting to multiply overlap by 2

3. Given predicted mask pixels: {1,2,3,4} and true mask pixels: {3,4,5,6}, what is the IoU score?

medium

A. 0.33

B. 0.25

C. 0.5

D. 0.4

Solution

1. Step 1: Calculate intersection and union

   Intersection = {3,4} size = 2; Union = {1,2,3,4,5,6} size = 6.

2. Step 2: Compute IoU

   IoU = Intersection / Union = 2 / 6 = 0.3333 (approx). Check options carefully.

3. Step 3: Re-check options

   0.33 is 0.33, closest to calculation. But 0.4 is 0.4, which is incorrect.

4. Final Answer:

   0.33 -> Option A

5. Quick Check:

   IoU = 2/6 = 0.33 [OK]

Hint: IoU = intersection size / union size [OK]

Common Mistakes:

• Counting union incorrectly
• Mixing intersection with union
• Rounding IoU incorrectly

4. You have this Python code to compute Dice score but it returns 0. Identify the error:

def dice_score(pred, true):
    intersection = len(pred & true)
    return intersection / (len(pred) + len(true))

pred = {1,2,3}
true = {2,3,4}
print(dice_score(pred, true))

medium

A. Wrong variable names in function

B. Using set intersection instead of union

C. Incorrect data type for masks

D. Missing multiplication by 2 in numerator

Solution

1. Step 1: Review Dice formula

   Dice = 2 * intersection / (sum of sizes). The code misses multiplying intersection by 2.

2. Step 2: Check code calculation

   Code returns intersection / sum, which is half the correct Dice score.

3. Final Answer:

   Missing multiplication by 2 in numerator -> Option D

4. Quick Check:

   Dice needs 2*intersection [OK]

Hint: Dice numerator must be 2 times intersection [OK]

Common Mistakes:

• Forgetting to multiply intersection by 2
• Confusing intersection with union
• Using wrong data types for masks

5. A segmentation model predicts a mask with 80 pixels, the true mask has 100 pixels, and their overlap is 60 pixels. What is the Dice score and IoU score respectively?

hard

A. Dice = 0.8, IoU = 0.7

B. Dice = 0.75, IoU = 0.6

C. Dice = 0.67, IoU = 0.5

D. Dice = 0.6, IoU = 0.75

Solution

1. Step 1: Calculate Dice score

   Dice = 2 * overlap / (predicted + true) = 2*60 / (80+100) = 120/180 = 0.6667 (approx). Check options carefully.

2. Step 2: Calculate IoU score

   IoU = overlap / union = 60 / (80 + 100 - 60) = 60 / 120 = 0.5.

3. Step 3: Re-check options

   Dice = 0.67, IoU = 0.5 matches Dice ~0.67 and IoU 0.5 exactly.

4. Final Answer:

   Dice = 0.67, IoU = 0.5 -> Option C

5. Quick Check:

   Dice = 2*60/180=0.67, IoU=60/120=0.5 [OK]

Hint: Dice doubles overlap; IoU divides by union [OK]

Common Mistakes:

• Using sum instead of union for IoU denominator
• Forgetting to multiply overlap by 2 for Dice
• Mixing Dice and IoU formulas