Fairness in face recognition in Computer Vision - Model Pipeline Trace

Model Pipeline - Fairness in face recognition

This pipeline shows how a face recognition system processes images to identify people while checking fairness across different groups. It highlights data handling, model training, and how fairness metrics improve over time.

Data Flow - 6 Stages

1Data Collection

10000 images x 3 channels (RGB) x 112 x 112 pixels→Gather face images from diverse groups with labels (person ID, demographic info)→10000 images x 3 channels x 112 x 112 pixels

Image of person A, labeled with ID=1, demographic=Group1

↓

2Preprocessing

10000 images x 3 channels x 112 x 112 pixels→Resize, normalize pixel values, and align faces→10000 images x 3 channels x 112 x 112 pixels (normalized)

Normalized pixel values between 0 and 1 for each image

↓

3Feature Extraction

10000 images x 3 channels x 112 x 112 pixels→Use CNN layers to extract 512-dimensional feature vectors→10000 samples x 512 features

[0.12, 0.45, ..., 0.33] feature vector for one face

↓

4Model Training

10000 samples x 512 features→Train classifier to predict person ID with fairness constraints→Trained model with learned weights

Model learns to map features to correct person IDs

↓

5Fairness Evaluation

Validation set features and labels→Calculate accuracy and fairness metrics (e.g., equal opportunity difference) across groups→Metrics report with accuracy and fairness scores per group

Group1 accuracy=92%, Group2 accuracy=85%, fairness gap=7%

↓

6Prediction

New face image x 3 channels x 112 x 112 pixels→Preprocess and extract features, then predict person ID→Predicted person ID with confidence score

Predicted ID=3 with 0.87 confidence

Training Trace - Epoch by Epoch

Loss
1.2 |*       
0.9 | **     
0.7 |  ***   
0.55|   **** 
0.45|    *****
     ----------------
      1  2  3  4  5  Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.60	Model starts learning, accuracy moderate, loss high
2	0.9	0.72	Loss decreases, accuracy improves, fairness gap still large
3	0.7	0.80	Better accuracy and lower loss, fairness constraints start to reduce gap
4	0.55	0.85	Model converging, fairness gap narrowing
5	0.45	0.88	Good accuracy and fairness balance achieved

Prediction Trace - 4 Layers

Layer 1: Input Image

Layer 2: Feature Extraction (CNN)

Layer 3: Classifier Layer

Layer 4: Prediction Output

Model Quiz - 3 Questions

What happens to the data shape after feature extraction?

A From images to 512-dimensional vectors

B From 512-dimensional vectors to images

C From images to raw pixel values

D From labels to images

Key Insight

This visualization shows that training a face recognition model with fairness constraints helps reduce performance gaps between demographic groups while maintaining good accuracy. Monitoring fairness metrics alongside accuracy is key to building responsible AI.