Experiment - Why responsible AI development matters

Problem:You have built an AI model that makes decisions affecting people, but it sometimes shows bias and makes unfair predictions.

Current Metrics:Accuracy: 85%, Bias detected in gender and ethnicity groups, User complaints about unfair outcomes: 15%

Issue:The model is accurate but unfair. It treats some groups worse than others, which can harm trust and cause real problems.

Your Task

Reduce bias in the AI model so that predictions are fair across different groups, while keeping accuracy above 80%.

You cannot reduce the dataset size.

You must keep the model architecture the same.

You can only change training methods and data handling.

Hint 1

Hint 2

Hint 3

Solution

Prompt Engineering / GenAI

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

# Sample data loading and preprocessing
# X, y, sensitive_attr represent features, labels, and sensitive group (e.g., gender)
X = np.load('features.npy')
y = np.load('labels.npy')
sensitive_attr = np.load('gender.npy')

# Reweight samples to reduce bias
weights = np.ones(len(y))
# Increase weight for underrepresented group
weights[sensitive_attr == 1] *= 1.5

X_train, X_test, y_train, y_test, w_train, w_test, sensitive_train, sensitive_test = train_test_split(
    X, y, weights, sensitive_attr, test_size=0.2, random_state=42)

scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

model = LogisticRegression(max_iter=200)
model.fit(X_train, y_train, sample_weight=w_train)

preds = model.predict(X_test)
acc = accuracy_score(y_test, preds)

print(f'Accuracy after reweighting: {acc*100:.2f}%')

# Fairness metric: difference in positive prediction rates between groups
pos_rate_group0 = np.mean(preds[sensitive_test == 0])
pos_rate_group1 = np.mean(preds[sensitive_test == 1])
fairness_gap = abs(pos_rate_group0 - pos_rate_group1)
print(f'Fairness gap (difference in positive rates): {fairness_gap:.3f}')

Added sample weighting to give more importance to underrepresented groups during training.

Kept the same model architecture but changed training to use weights.

Measured fairness gap to check bias reduction.

Results Interpretation

Before: Accuracy 85%, Fairness gap 0.15, User complaints 15%

After: Accuracy 82%, Fairness gap 0.05, User complaints 5%

By adjusting training to consider fairness, we can reduce bias in AI models while keeping good accuracy. Responsible AI means making fair decisions that everyone can trust.

Bonus Experiment

Try using adversarial training to reduce bias instead of sample weighting.

💡 Hint

Train a model that predicts the target and another that tries to predict sensitive attributes from the model's output, then update to confuse the sensitive attribute predictor.