Bias detection and mitigation in ML Python

Introduction

Bias detection and mitigation help make sure AI models treat everyone fairly. It stops models from making unfair decisions based on gender, race, or other traits.

When building a hiring tool to avoid unfairly rejecting candidates from certain groups.

When creating a loan approval system to ensure fair treatment for all applicants.

When developing a facial recognition system to reduce errors for different skin tones.

When analyzing medical data to prevent biased health predictions.

When training chatbots to avoid offensive or biased responses.

Syntax

ML Python

1. Collect data and check for bias using fairness metrics.
2. Use techniques like re-sampling, re-weighting, or adversarial training to reduce bias.
3. Evaluate the model again to confirm bias is reduced.

Bias detection uses fairness metrics like demographic parity or equal opportunity.

Mitigation can happen before, during, or after training the model.

Examples

This code checks if two groups get positive predictions at similar rates.

ML Python

# Example: Checking bias with demographic parity
from sklearn.metrics import confusion_matrix

# Calculate positive rates for groups
positive_rate_group1 = sum(predictions[group1]) / len(group1)
positive_rate_group2 = sum(predictions[group2]) / len(group2)

This code gives more importance to underrepresented group samples during training.

ML Python

# Example: Mitigation by re-weighting samples
weights = [1.5 if x in underrepresented_group else 1.0 for x in data]
model.fit(data, labels, sample_weight=weights)

Sample Model

This example shows how to detect bias by comparing positive prediction rates between two groups. Then it applies re-weighting to reduce bias and checks the results.

ML Python

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

# Sample data: features and labels
X = np.array([[1, 20], [2, 22], [3, 21], [4, 23], [5, 24], [6, 25]])
y = np.array([0, 0, 1, 1, 1, 0])

# Sensitive attribute (e.g., group membership)
groups = np.array([0, 0, 1, 1, 1, 0])

# Train model without mitigation
model = LogisticRegression()
model.fit(X, y)
preds = model.predict(X)
accuracy = accuracy_score(y, preds)

# Calculate positive prediction rates per group
pos_rate_group0 = preds[groups == 0].mean()
pos_rate_group1 = preds[groups == 1].mean()

# Mitigation: re-weight samples to balance groups
weights = np.where(groups == 0, 1.0, 1.5)
model_mitigated = LogisticRegression()
model_mitigated.fit(X, y, sample_weight=weights)
preds_mitigated = model_mitigated.predict(X)
accuracy_mitigated = accuracy_score(y, preds_mitigated)

pos_rate_group0_m = preds_mitigated[groups == 0].mean()
pos_rate_group1_m = preds_mitigated[groups == 1].mean()

print(f"Before mitigation accuracy: {accuracy:.2f}")
print(f"Positive rate group 0 before: {pos_rate_group0:.2f}")
print(f"Positive rate group 1 before: {pos_rate_group1:.2f}")
print(f"After mitigation accuracy: {accuracy_mitigated:.2f}")
print(f"Positive rate group 0 after: {pos_rate_group0_m:.2f}")
print(f"Positive rate group 1 after: {pos_rate_group1_m:.2f}")

Important Notes

Bias detection is not just about accuracy; fairness metrics are key.

Mitigation may slightly reduce accuracy but improves fairness.

Always test models on diverse data to catch hidden biases.

Summary

Bias detection finds unfair treatment in AI models.

Mitigation techniques help models treat groups more equally.

Checking fairness is important for trustworthy AI.