AI governance frameworks in Prompt Engineering / GenAI - ML Experiment: Train & Evaluate

Experiment - AI governance frameworks

Problem:You have developed an AI model that makes decisions affecting users. However, there is no clear system to ensure the AI behaves fairly, transparently, and safely.

Current Metrics:No formal metrics; incidents of biased decisions reported by users; lack of transparency in AI decisions.

Issue:The AI model lacks governance controls, leading to risks of unfair outcomes, lack of accountability, and potential harm to users.

Your Task

Design and implement an AI governance framework that ensures fairness, transparency, and accountability in the AI model's decisions.

You cannot change the AI model's core algorithm.

You must use explainability tools and monitoring techniques.

The framework should be easy to understand for non-technical stakeholders.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Prompt Engineering / GenAI

import shap
import numpy as np
import pandas as pd
from sklearn.metrics import accuracy_score, confusion_matrix

# Assume model and data are preloaded: model, X_test, y_test

# Step 1: Evaluate fairness by checking confusion matrix for different groups
# For example, check performance for group A and group B

def group_performance(model, X, y, group_mask):
    preds = model.predict(X[group_mask])
    acc = accuracy_score(y[group_mask], preds)
    cm = confusion_matrix(y[group_mask], preds)
    return acc, cm

# Step 2: Explain predictions using SHAP
explainer = shap.Explainer(model, X_test)
shap_values = explainer(X_test)

# Step 3: Create simple monitoring metrics
accuracy = accuracy_score(y_test, model.predict(X_test))

# Step 4: Output summary report
report = {
    'overall_accuracy': accuracy,
    'group_A_performance': group_performance(model, X_test, y_test, X_test['group'] == 'A'),
    'group_B_performance': group_performance(model, X_test, y_test, X_test['group'] == 'B'),
    'shap_summary': 'SHAP summary plot generated (not stored in report)'
}

shap.summary_plot(shap_values, X_test, show=False)

print('AI Governance Report:', report)

Added fairness evaluation by comparing accuracy and confusion matrices for different user groups.

Integrated SHAP explainability to clarify how features influence model decisions.

Set up simple monitoring metrics like overall accuracy.

Created a summary report to communicate AI behavior transparently.

Results Interpretation

Before: No fairness checks, no transparency, user complaints about bias.
After: Fairness metrics show balanced accuracy across groups, explanations clarify decisions, monitoring enables ongoing oversight.

Implementing AI governance frameworks helps detect and reduce bias, improves transparency, and builds trust by making AI decisions understandable and accountable.

Bonus Experiment

Now try integrating automated alerts that notify stakeholders when fairness metrics drop below a threshold.

💡 Hint

Use monitoring tools to track metrics continuously and trigger email or dashboard alerts when anomalies occur.

Practice

1. What is the main purpose of an AI governance framework?

easy

A. To increase the speed of AI model training

B. To improve the graphical user interface of AI apps

C. To reduce the cost of AI hardware

D. To guide safe and fair use of AI systems

Solution

1. Step 1: Understand the role of AI governance frameworks

   AI governance frameworks are designed to ensure AI is used responsibly and ethically.

2. Step 2: Identify the main goal

   The main goal is to guide safe and fair use, preventing harm and building trust.

3. Final Answer:

   To guide safe and fair use of AI systems -> Option D

4. Quick Check:

   Purpose of AI governance = safe and fair use [OK]

Hint: Focus on safety and fairness in AI use [OK]

Common Mistakes:

• Confusing governance with technical optimization
• Thinking governance is about cost or speed
• Ignoring ethical and safety aspects

2. Which of the following is a correct component of an AI governance framework?

easy

A. Faster GPU hardware

B. Policies and processes for AI use

C. New programming languages

D. User interface design templates

Solution

1. Step 1: Recall components of AI governance frameworks

   They include principles, policies, processes, roles, and tools to manage AI responsibly.

2. Step 2: Match options to components

   Only policies and processes relate directly to governance frameworks.

3. Final Answer:

   Policies and processes for AI use -> Option B

4. Quick Check:

   Governance components = policies and processes [OK]

Hint: Look for management and rules, not tech specs [OK]

Common Mistakes:

• Choosing hardware or software unrelated to governance
• Confusing governance with development tools
• Ignoring the role of policies

3. Consider this code snippet representing a simple AI governance check in Python:

def check_fairness(data):
    if 'bias' in data:
        return 'Unfair AI detected'
    else:
        return 'AI is fair'

result = check_fairness(['accuracy', 'bias'])
print(result)

What will be the output?

medium

A. Unfair AI detected

B. Error: 'bias' not defined

C. AI is fair

D. None

Solution

1. Step 1: Analyze the function logic

   The function checks if the string 'bias' is in the input list. If yes, it returns 'Unfair AI detected'.

2. Step 2: Check the input data

   The input list contains 'accuracy' and 'bias', so 'bias' is present.

3. Final Answer:

   Unfair AI detected -> Option A

4. Quick Check:

   Presence of 'bias' triggers unfair AI message [OK]

Hint: Check if 'bias' is in the list to decide output [OK]

Common Mistakes:

• Assuming 'bias' is a variable, not a string
• Ignoring the if condition logic
• Thinking the function returns None

4. The following code is intended to check if an AI model meets governance standards by verifying if it has a 'transparency' attribute. Identify the error:

class AIModel:
    def __init__(self, transparency):
        self.transparency = transparency

def check_governance(model):
    if model.transparency == True:
        return 'Governance passed'
    else:
        return 'Governance failed'

model = AIModel('yes')
print(check_governance(model))

medium

A. The transparency attribute should be a boolean, not a string

B. The method check_governance is missing a return statement

C. The class AIModel lacks a constructor

D. The print statement is outside the function

Solution

1. Step 1: Understand the attribute type check

   The function compares model.transparency to True (boolean).

2. Step 2: Check the attribute value in the instance

   The model is created with 'yes' (string), not True (boolean), so the condition fails.

3. Final Answer:

   The transparency attribute should be a boolean, not a string -> Option A

4. Quick Check:

   Type mismatch causes governance check failure [OK]

Hint: Match attribute types with condition checks [OK]

Common Mistakes:

• Ignoring type mismatch between string and boolean
• Thinking missing return causes error here
• Confusing class constructor presence

5. You are designing an AI governance framework for a healthcare AI system. Which combination of components best ensures ethical use and accountability?

hard

A. High accuracy metrics, cloud deployment, and automated updates

B. Faster model training, open-source code, and user-friendly UI

C. Clear policies, regular audits, and defined roles for oversight

D. Large datasets, complex algorithms, and minimal documentation

Solution

1. Step 1: Identify key governance needs in healthcare AI

   Ethical use and accountability require clear rules, monitoring, and responsible roles.

2. Step 2: Evaluate options for governance components

   Only Clear policies, regular audits, and defined roles for oversight includes policies, audits, and roles which align with governance goals.

3. Final Answer:

   Clear policies, regular audits, and defined roles for oversight -> Option C

4. Quick Check:

   Governance needs policies + audits + roles [OK]

Hint: Governance = policies + audits + roles, not tech features [OK]

Common Mistakes:

• Confusing governance with technical performance
• Ignoring the need for oversight roles
• Choosing options focused on speed or complexity