FCN (Fully Convolutional Network) in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - FCN (Fully Convolutional Network)

Which metric matters for FCN and WHY

Fully Convolutional Networks (FCNs) are mainly used for image segmentation. This means the model labels each pixel in an image. So, the key metrics measure how well the model predicts pixel classes.

Common metrics:

• Intersection over Union (IoU): Measures overlap between predicted and true pixel areas. Higher IoU means better segmentation.
• Pixel Accuracy: Percentage of pixels correctly classified. Simple but can be misleading if classes are imbalanced.
• Dice Coefficient (F1 score for pixels): Balances precision and recall for pixel classification.

These metrics help us understand how well the FCN segments images, which is the main goal.

Confusion matrix for pixel classification

For segmentation, the confusion matrix counts pixels:

      | Predicted Positive | Predicted Negative |
      |--------------------|--------------------|
      | True Positive (TP)  | False Negative (FN) |
      | False Positive (FP) | True Negative (TN)  |
    

Example for one class (e.g., 'car' pixels):

      TP = 8000 pixels (correctly predicted as car)
      FP = 2000 pixels (wrongly predicted as car)
      FN = 1000 pixels (car pixels missed)
      TN = 89000 pixels (correctly predicted as not car)
    

Total pixels = TP + FP + FN + TN = 8000 + 2000 + 1000 + 89000 = 100000 pixels

Precision vs Recall tradeoff in FCN segmentation

Precision tells us how many pixels predicted as a class are actually correct.

Recall tells us how many true pixels of a class were found by the model.

Example:

• High precision but low recall: Model is very sure about predicted pixels but misses many true pixels. Good if false positives are costly.
• High recall but low precision: Model finds most true pixels but also predicts many wrong pixels. Good if missing pixels is costly.

For medical image segmentation, high recall is important to not miss any diseased area. For autonomous driving, high precision avoids false alarms.

What good vs bad metric values look like for FCN

Good segmentation:

• IoU above 0.7 (70%) means strong overlap between prediction and truth.
• Pixel accuracy above 90% if classes are balanced.
• Dice coefficient above 0.8 shows good balance of precision and recall.

Bad segmentation:

• IoU below 0.4 means poor overlap, many pixels misclassified.
• Pixel accuracy can be misleadingly high if background dominates, so low IoU or Dice is a warning.
• Dice coefficient below 0.5 means model struggles to find correct pixels.

Common pitfalls in FCN metric evaluation

• Accuracy paradox: High pixel accuracy can happen if background pixels dominate, hiding poor class predictions.
• Ignoring class imbalance: Small objects may have low IoU but be important; metrics should consider this.
• Data leakage: Using test images in training inflates metrics falsely.
• Overfitting: Very high training metrics but low test metrics means model memorizes training images, not generalizing.

Self-check question

Your FCN model has 98% pixel accuracy but only 12% recall on a small but important class (e.g., tumor pixels). Is this good for production?

Answer: No. The model misses 88% of true tumor pixels. High accuracy is misleading because most pixels are background. Low recall means many tumors go undetected, which is dangerous.

Key Result

For FCNs, IoU and Dice coefficient best measure segmentation quality, while pixel accuracy can be misleading due to class imbalance.

Practice

1. What is the main advantage of a Fully Convolutional Network (FCN) compared to a traditional CNN for image tasks?

easy

A. It can label every pixel in the image, preserving spatial information.

B. It uses fully connected layers to improve classification accuracy.

C. It reduces the image size to a single vector for faster processing.

D. It only works with grayscale images for simplicity.

Solution

1. Step 1: Understand FCN structure

   FCNs replace fully connected layers with convolution layers to keep spatial info.

2. Step 2: Compare with traditional CNNs

   Traditional CNNs use fully connected layers that lose pixel location details, FCNs keep pixel-wise info.

3. Final Answer:

   It can label every pixel in the image, preserving spatial information. -> Option A

4. Quick Check:

   FCN labels pixels = B [OK]

Hint: FCNs keep pixel info; CNNs lose it with fully connected layers [OK]

Common Mistakes:

• Thinking FCNs use fully connected layers
• Assuming FCNs reduce image to vector
• Believing FCNs only work on grayscale

2. Which of the following is the correct way to replace a fully connected layer with a convolutional layer in an FCN?

easy

A. nn.Linear(in_features, out_features)

B. nn.ReLU()

C. nn.Conv2d(in_channels, out_channels, kernel_size=1)

D. nn.MaxPool2d(kernel_size=2)

Solution

1. Step 1: Identify layer types

   Fully connected layers are replaced by 1x1 convolution layers to keep spatial size.

2. Step 2: Match correct PyTorch layer

   nn.Conv2d with kernel size 1 acts like a fully connected layer per pixel.

3. Final Answer:

   nn.Conv2d(in_channels, out_channels, kernel_size=1) -> Option C

4. Quick Check:

   1x1 conv replaces FC layer = D [OK]

Hint: Use 1x1 conv to replace fully connected layers [OK]

Common Mistakes:

• Choosing nn.Linear which is fully connected
• Confusing pooling or activation layers as replacements
• Using wrong kernel sizes

3. Given this PyTorch FCN snippet, what is the output shape if the input is (1, 3, 64, 64)?

import torch
import torch.nn as nn

class SimpleFCN(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding=1)
        self.conv2 = nn.Conv2d(16, 2, 1)
    def forward(self, x):
        x = self.conv1(x)
        x = self.conv2(x)
        return x

model = SimpleFCN()
input_tensor = torch.randn(1, 3, 64, 64)
output = model(input_tensor)
print(output.shape)

medium

A. (1, 2, 64, 64)

B. (1, 2, 62, 62)

C. (1, 16, 64, 64)

D. (1, 3, 64, 64)

Solution

1. Step 1: Analyze first conv layer

   Conv2d(3,16,3,padding=1) keeps spatial size 64x64, output shape (1,16,64,64).

2. Step 2: Analyze second conv layer

   Conv2d(16,2,1) with kernel size 1 keeps spatial size, output shape (1,2,64,64).

3. Final Answer:

   (1, 2, 64, 64) -> Option A

4. Quick Check:

   Padding keeps size; 1x1 conv keeps size = A [OK]

Hint: Padding keeps size; 1x1 conv keeps spatial dims [OK]

Common Mistakes:

• Ignoring padding effect on size
• Assuming kernel 3 reduces size without padding
• Confusing channel and spatial dimensions

4. You have an FCN model but the output size is smaller than the input image size. What is the most likely cause?

medium

A. Using 1x1 convolution layers increases spatial size.

B. Using convolution layers without padding reduces spatial size.

C. Using fully connected layers preserves spatial size.

D. Using batch normalization layers reduces spatial size.

Solution

1. Step 1: Understand convolution padding effect

   Convolution without padding shrinks spatial dimensions because kernel slides inside image.

2. Step 2: Identify cause of smaller output

   If output is smaller, likely padding is missing in conv layers causing shrinkage.

3. Final Answer:

   Using convolution layers without padding reduces spatial size. -> Option B

4. Quick Check:

   No padding shrinks output = A [OK]

Hint: No padding in conv layers shrinks output size [OK]

Common Mistakes:

• Thinking 1x1 conv changes size
• Believing batch norm affects size
• Assuming fully connected layers keep spatial size

5. You want to build an FCN for semantic segmentation on 128x128 images with 3 classes. Which architecture choice will produce output matching input size with class scores per pixel?

hard

A. Use fully connected layers after conv layers to classify whole image.

B. Use conv layers without padding and max pooling layers to reduce size.

C. Use conv layers with stride=2 and no upsampling layers.

D. Use conv layers with padding=1 and 1x1 conv at end, no pooling layers.

Solution

1. Step 1: Maintain spatial size

   Padding=1 with kernel size 3 keeps spatial size; no pooling avoids shrinking.

2. Step 2: Output class scores per pixel

   Use 1x1 conv with out_channels=3 to get class scores per pixel matching input size.

3. Final Answer:

   Use conv layers with padding=1 and 1x1 conv at end, no pooling layers. -> Option D

4. Quick Check:

   Padding + 1x1 conv = pixel-wise classes = C [OK]

Hint: Keep padding and no pooling for same size output [OK]

Common Mistakes:

• Using pooling reduces output size
• Using fully connected layers loses pixel info
• Using stride>1 without upsampling shrinks output