Fairness in face recognition in Computer Vision - Model Pipeline Trace

Model Pipeline - Fairness in face recognition

This pipeline shows how a face recognition system processes images to identify people while checking fairness across different groups. It highlights data handling, model training, and how fairness metrics improve over time.

Data Flow - 6 Stages

1Data Collection

10000 images x 3 channels (RGB) x 112 x 112 pixels→Gather face images from diverse groups with labels (person ID, demographic info)→10000 images x 3 channels x 112 x 112 pixels

Image of person A, labeled with ID=1, demographic=Group1

↓

2Preprocessing

10000 images x 3 channels x 112 x 112 pixels→Resize, normalize pixel values, and align faces→10000 images x 3 channels x 112 x 112 pixels (normalized)

Normalized pixel values between 0 and 1 for each image

↓

3Feature Extraction

10000 images x 3 channels x 112 x 112 pixels→Use CNN layers to extract 512-dimensional feature vectors→10000 samples x 512 features

[0.12, 0.45, ..., 0.33] feature vector for one face

↓

4Model Training

10000 samples x 512 features→Train classifier to predict person ID with fairness constraints→Trained model with learned weights

Model learns to map features to correct person IDs

↓

5Fairness Evaluation

Validation set features and labels→Calculate accuracy and fairness metrics (e.g., equal opportunity difference) across groups→Metrics report with accuracy and fairness scores per group

Group1 accuracy=92%, Group2 accuracy=85%, fairness gap=7%

↓

6Prediction

New face image x 3 channels x 112 x 112 pixels→Preprocess and extract features, then predict person ID→Predicted person ID with confidence score

Predicted ID=3 with 0.87 confidence

Training Trace - Epoch by Epoch

Loss
1.2 |*       
0.9 | **     
0.7 |  ***   
0.55|   **** 
0.45|    *****
     ----------------
      1  2  3  4  5  Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.60	Model starts learning, accuracy moderate, loss high
2	0.9	0.72	Loss decreases, accuracy improves, fairness gap still large
3	0.7	0.80	Better accuracy and lower loss, fairness constraints start to reduce gap
4	0.55	0.85	Model converging, fairness gap narrowing
5	0.45	0.88	Good accuracy and fairness balance achieved

Prediction Trace - 4 Layers

Layer 1: Input Image

Layer 2: Feature Extraction (CNN)

Layer 3: Classifier Layer

Layer 4: Prediction Output

Model Quiz - 3 Questions

What happens to the data shape after feature extraction?

A From images to 512-dimensional vectors

B From 512-dimensional vectors to images

C From images to raw pixel values

D From labels to images

Key Insight

This visualization shows that training a face recognition model with fairness constraints helps reduce performance gaps between demographic groups while maintaining good accuracy. Monitoring fairness metrics alongside accuracy is key to building responsible AI.

Practice

1.

What does fairness in face recognition mainly aim to achieve?

easy

A. More complex model architecture

B. Faster processing speed

C. Higher resolution images

D. Equal accuracy for all demographic groups

Solution

1. Step 1: Understand fairness goal

   Fairness means the model should work equally well for all groups, not just some.

2. Step 2: Identify fairness metric

   Accuracy or error rates should be similar across different demographic groups.

3. Final Answer:

   Equal accuracy for all demographic groups -> Option D

4. Quick Check:

   Fairness = Equal accuracy [OK]

Hint: Fairness means equal results for everyone [OK]

Common Mistakes:

• Thinking fairness means faster models
• Confusing fairness with image quality
• Assuming complex models are always fair

2.

Which of the following is the correct way to check fairness in a face recognition model?

metrics = {'group_A': 0.92, 'group_B': 0.85}
# What should we compare?

easy

A. Only check metrics['group_A']

B. Compare metrics['group_A'] and metrics['group_B'] for equality

C. Ignore metrics and check model size

D. Compare metrics['group_A'] with a random number

Solution

1. Step 1: Identify fairness check

   Fairness requires comparing performance metrics across groups.

2. Step 2: Apply comparison

   Compare accuracy or error rates between group_A and group_B to find bias.

3. Final Answer:

   Compare metrics['group_A'] and metrics['group_B'] for equality -> Option B

4. Quick Check:

   Fairness check = Compare group metrics [OK]

Hint: Compare group metrics to check fairness [OK]

Common Mistakes:

• Checking only one group
• Ignoring metrics and focusing on model size
• Comparing to unrelated values

3.

Consider this Python code snippet evaluating fairness metrics:

group_accuracies = {'A': 0.90, 'B': 0.75, 'C': 0.88}
threshold = 0.80
biased_groups = [g for g, acc in group_accuracies.items() if acc < threshold]
print(biased_groups)

What is the output?

medium

A. ['B']

B. ['A', 'B']

C. ['C']

D. []

Solution

1. Step 1: Understand the code logic

   The code collects groups with accuracy less than 0.80 into biased_groups.

2. Step 2: Check each group's accuracy

   Group A: 0.90 > 0.80 (not biased), B: 0.75 < 0.80 (biased), C: 0.88 > 0.80 (not biased)

3. Final Answer:

   ['B'] -> Option A

4. Quick Check:

   Only group B accuracy < threshold [OK]

Hint: Filter groups with accuracy below threshold [OK]

Common Mistakes:

• Including groups with accuracy above threshold
• Misreading comparison operator
• Confusing list comprehension output

4.

Find the error in this fairness evaluation code snippet:

metrics = {'group1': 0.85, 'group2': 0.80}
threshold = 0.82
biased = [g for g, v in metrics if v < threshold]
print(biased)

medium

A. Missing .items() when iterating over dictionary

B. Wrong comparison operator

C. Threshold value is too high

D. Print statement syntax error

Solution

1. Step 1: Identify dictionary iteration error

   Iterating over a dictionary directly gives keys, not key-value pairs.

2. Step 2: Fix iteration to use .items()

   Use metrics.items() to get (key, value) pairs for comparison.

3. Final Answer:

   Missing .items() when iterating over dictionary -> Option A

4. Quick Check:

   Dictionary iteration needs .items() [OK]

Hint: Use .items() to get key-value pairs from dict [OK]

Common Mistakes:

• Iterating dict keys instead of items
• Changing threshold unnecessarily
• Assuming print syntax is wrong

5.

You have a face recognition model with accuracy 0.95 on group X and 0.70 on group Y. Which approach best improves fairness?

hard

A. Ignore group Y and focus on group X

B. Increase model complexity without changing data

C. Collect more balanced training data including group Y

D. Reduce accuracy on group X to match group Y

Solution

1. Step 1: Identify fairness problem

   Model performs worse on group Y, showing bias.

2. Step 2: Choose best fairness improvement

   Balanced data helps model learn features for all groups equally.

3. Step 3: Evaluate other options

   Increasing complexity alone may not fix bias; ignoring group Y is unfair; reducing group X accuracy is not ideal.

4. Final Answer:

   Collect more balanced training data including group Y -> Option C

5. Quick Check:

   Balanced data improves fairness [OK]

Hint: Balance training data to reduce bias [OK]

Common Mistakes:

• Thinking model complexity fixes bias alone
• Ignoring underperforming groups
• Lowering accuracy on better groups