Semantic segmentation vs instance segmentation in Computer Vision - Model Approaches Compared

Model Pipeline - Semantic segmentation vs instance segmentation

This pipeline compares two popular image segmentation tasks: semantic segmentation and instance segmentation. Both divide an image into meaningful parts, but semantic segmentation labels each pixel by class, while instance segmentation also separates individual objects of the same class.

Data Flow - 5 Stages

1Input Image

1 image x 256 height x 256 width x 3 channels→Raw color image input→1 image x 256 height x 256 width x 3 channels

A photo showing multiple dogs and cats in a park

↓

2Preprocessing

1 image x 256 x 256 x 3→Resize and normalize pixel values to 0-1→1 image x 256 x 256 x 3

Image pixels scaled from 0-255 to 0-1

↓

3Feature Extraction

1 image x 256 x 256 x 3→Convolutional layers extract visual features→1 image x 64 x 64 x 128 features

Edges, textures, and shapes detected in image

↓

4Semantic Segmentation Head

1 image x 64 x 64 x 128→Predict class label for each pixel→1 image x 256 x 256 x number_of_classes

Pixels labeled as 'dog', 'cat', 'background'

↓

5Instance Segmentation Head

1 image x 64 x 64 x 128→Detect and segment individual object instances→List of masks with shape 256 x 256 per instance

Separate masks for each dog and cat in image

Training Trace - Epoch by Epoch

Loss:
1.2 |************
0.8 |********
0.5 |*****
0.35|****
0.25|***
Epochs ->

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.45	Initial training with high loss and low accuracy
5	0.8	0.65	Model starts learning to segment classes better
10	0.5	0.80	Good improvement in pixel classification accuracy
15	0.35	0.88	Model accurately segments most objects
20	0.25	0.92	Converged with high accuracy and low loss

Prediction Trace - 5 Layers

Layer 1: Input Image

Layer 2: Feature Extraction

Layer 3: Semantic Segmentation Head

Layer 4: Instance Segmentation Head

Layer 5: Final Output

Model Quiz - 3 Questions

What is the main difference between semantic and instance segmentation?

A Semantic segmentation labels each pixel by class, instance segmentation also separates individual objects

B Semantic segmentation detects objects, instance segmentation classifies images

C Instance segmentation only works on black and white images

D Semantic segmentation groups all images into one class

Key Insight

Semantic segmentation assigns a class to every pixel, useful for understanding overall scene layout. Instance segmentation goes further by separating each object, enabling detailed object-level analysis. Training shows loss decreasing and accuracy increasing, indicating the model learns to segment images well.