Bias and fairness in NLP - ML Experiment: Train & Evaluate

Experiment - Bias and fairness in NLP

Problem:You have a sentiment analysis model trained on movie reviews. The model shows good overall accuracy but performs worse on reviews written by certain demographic groups, indicating bias.

Current Metrics:Overall accuracy: 88%, Accuracy on group A: 90%, Accuracy on group B: 75%

Issue:The model is biased against group B, showing lower accuracy and fairness issues.

Your Task

Reduce bias so that accuracy on group B improves to at least 85% while maintaining overall accuracy above 85%.

You cannot collect new data.

You must use the existing dataset and model architecture.

You can only modify training procedures or add fairness techniques.

Hint 1

Hint 2

Hint 3

Solution

NLP

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, LSTM, Dense, Dropout
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences

# Sample data setup (simplified)
texts = ["good movie", "bad movie", "excellent film", "terrible film"] * 100
labels = [1, 0, 1, 0] * 100
# Group labels: 0 for group A, 1 for group B
groups = np.array([0, 0, 1, 1] * 100)

# Tokenize texts
max_words = 1000
tokenizer = Tokenizer(num_words=max_words)
tokenizer.fit_on_texts(texts)
sequences = tokenizer.texts_to_sequences(texts)
data = pad_sequences(sequences, maxlen=5)

# Create sample weights to reduce bias: give higher weight to group B samples
sample_weights = np.where(groups == 1, 2.0, 1.0)

# Split data
X_train, X_val, y_train, y_val, sw_train, sw_val, groups_train, groups_val = train_test_split(
    data, labels, sample_weights, groups, test_size=0.2, random_state=42, stratify=labels)

# Build simple model
model = Sequential([
    Embedding(max_words, 16, input_length=5),
    LSTM(16),
    Dropout(0.5),
    Dense(1, activation='sigmoid')
])

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Train with sample weights to reduce bias
model.fit(X_train, y_train, sample_weight=sw_train, epochs=10, batch_size=16, validation_data=(X_val, y_val, sw_val))

# Evaluate overall accuracy
val_preds = (model.predict(X_val) > 0.5).astype(int).flatten()
accuracy_overall = accuracy_score(y_val, val_preds) * 100

# Evaluate accuracy on group B in validation
group_b_mask = (groups_val == 1)
accuracy_group_b = accuracy_score(y_val[group_b_mask], val_preds[group_b_mask]) * 100

print(f"Validation accuracy overall: {accuracy_overall:.2f}%")
print(f"Validation accuracy group B: {accuracy_group_b:.2f}%")

Added sample weights to give more importance to group B samples during training.

Included dropout layer to reduce overfitting and improve generalization.

Kept model architecture simple to focus on fairness improvement.

Added sample weights to validation_data in model.fit to ensure correct evaluation.

Results Interpretation

Before: Overall accuracy: 88%, Group B accuracy: 75%

After: Overall accuracy: 87%, Group B accuracy: 86%

Using sample weighting during training can reduce bias by making the model pay more attention to underrepresented or disadvantaged groups, improving fairness without sacrificing much overall accuracy.

Bonus Experiment

Try using adversarial training to remove demographic information from the model's internal representation to further reduce bias.

💡 Hint

Add an adversarial network that tries to predict group labels from model features and train the main model to fool it.

Practice

1. What does bias in NLP models usually mean?

easy

A. The model always predicts correctly

B. Unfair treatment of some groups by the model

C. The model runs faster on some data

D. The model uses more memory for some inputs

Solution

1. Step 1: Understand the meaning of bias in NLP

   Bias refers to when a model treats some groups unfairly, often due to skewed training data or design.

2. Step 2: Compare options to definition

   Only Unfair treatment of some groups by the model describes unfair treatment, which matches the definition of bias in NLP.

3. Final Answer:

   Unfair treatment of some groups by the model -> Option B

4. Quick Check:

   Bias = Unfair treatment [OK]

Hint: Bias means unfairness in model predictions [OK]

Common Mistakes:

• Confusing bias with model speed or memory use
• Thinking bias means always correct predictions

2. Which of the following is the correct way to check fairness in an NLP model?

easy

A. Count the number of layers in the model

B. Check if the model uses GPU acceleration

C. Compare accuracy across different demographic groups

D. Measure the model's training time

Solution

1. Step 1: Identify fairness checking methods

   Fairness is checked by comparing performance metrics like accuracy across groups to ensure equal treatment.

2. Step 2: Evaluate options

   Only Compare accuracy across different demographic groups relates to fairness by comparing accuracy across groups; others are unrelated to fairness.

3. Final Answer:

   Compare accuracy across different demographic groups -> Option C

4. Quick Check:

   Fairness check = Compare accuracy by group [OK]

Hint: Fairness means equal accuracy for all groups [OK]

Common Mistakes:

• Confusing fairness with model speed or architecture
• Ignoring group-based performance differences

3. Consider this Python code snippet checking fairness metrics:

group_accuracies = {'groupA': 0.85, 'groupB': 0.60}
if abs(group_accuracies['groupA'] - group_accuracies['groupB']) > 0.2:
    print('Fairness issue detected')
else:
    print('No fairness issue')

What will this code print?

medium

A. KeyError

B. No fairness issue

C. SyntaxError

D. Fairness issue detected

Solution

1. Step 1: Calculate difference in accuracies

   The difference is |0.85 - 0.60| = 0.25, which is greater than 0.2.

2. Step 2: Evaluate the if condition

   Since 0.25 > 0.2, the condition is true, so it prints 'Fairness issue detected'.

3. Final Answer:

   Fairness issue detected -> Option D

4. Quick Check:

   Difference 0.25 > 0.2 = Fairness issue [OK]

Hint: Check if accuracy difference > threshold for fairness [OK]

Common Mistakes:

• Miscomputing the absolute difference
• Confusing greater than with less than
• Expecting syntax or key errors

4. This code tries to calculate fairness but has a bug:

metrics = {'group1': {'accuracy': 0.9}, 'group2': {'accuracy': 0.85}}
diff = metrics['group1']['accuracy'] - metrics['group3']['accuracy']
if abs(diff) > 0.05:
    print('Bias detected')

What is the error and how to fix it?

medium

A. KeyError because 'group3' does not exist; fix by checking keys first

B. SyntaxError due to missing colon; fix by adding colon

C. TypeError because accuracy is not a number; fix by converting to float

D. No error; code runs fine

Solution

1. Step 1: Identify the error cause

   The code accesses metrics['group3'], which is not in the dictionary, causing a KeyError.

2. Step 2: Suggest fix

   Check if 'group3' exists in metrics before accessing or handle missing keys to avoid error.

3. Final Answer:

   KeyError because 'group3' does not exist; fix by checking keys first -> Option A

4. Quick Check:

   Missing key access = KeyError [OK]

Hint: Check dictionary keys before access to avoid KeyError [OK]

Common Mistakes:

• Assuming all keys exist without checking
• Confusing KeyError with SyntaxError or TypeError

5. You have an NLP sentiment model that predicts positive or negative sentiment. You notice it predicts positive sentiment 90% for group A but only 60% for group B, though both groups have similar real sentiment. What is the best way to improve fairness?

hard

A. Collect more balanced training data including both groups equally

B. Increase model size to improve overall accuracy

C. Use a faster optimizer to train the model

D. Remove group B data from training to avoid confusion

Solution

1. Step 1: Understand the fairness problem

   The model predicts differently for groups with similar real sentiment, indicating bias likely from unbalanced data.

2. Step 2: Choose the best fix

   Collecting balanced data ensures the model learns equally from both groups, improving fairness.

3. Final Answer:

   Collect more balanced training data including both groups equally -> Option A

4. Quick Check:

   Balanced data improves fairness [OK]

Hint: Balanced data helps fix bias in predictions [OK]

Common Mistakes:

• Thinking bigger models fix bias automatically
• Ignoring data imbalance as cause of unfairness
• Removing data from minority groups