Responsible AI practices in MLOps - Commands & Configuration

Introduction

Responsible AI practices help ensure that AI models are fair, safe, and trustworthy. They guide how to build and deploy AI systems that respect privacy, avoid bias, and provide clear explanations.

When you want to check if your AI model treats all groups fairly before deployment

When you need to track and explain AI decisions to users or regulators

When you want to monitor AI model behavior continuously to catch errors or bias

When you must protect sensitive data used in AI training and predictions

When you want to document AI model development steps for transparency

Commands

This command runs an MLflow project that includes responsible AI checks like fairness and explainability during model training.

Terminal

mlflow run https://github.com/mlflow/mlflow-example.git -P alpha=0.5

Expected Output

2024/06/01 12:00:00 INFO mlflow.projects: === Running command 'python train.py --alpha 0.5' in run with ID '123abc' ===

Training model with alpha=0.5

Logging metrics and artifacts

Run completed successfully

→

-P - Passes parameters to the MLflow project, here setting model hyperparameter alpha

Starts the MLflow tracking UI to visualize model metrics, fairness reports, and explanations collected during runs.

Terminal

mlflow ui

Expected Output

2024/06/01 12:01:00 INFO mlflow.server: Starting MLflow UI at http://127.0.0.1:5000

Serves the trained model with responsible AI features enabled, allowing safe and explainable predictions via REST API.

Terminal

mlflow models serve -m runs:/123abc/model -p 1234

Expected Output

2024/06/01 12:02:00 INFO mlflow.models: Serving model from run 123abc on port 1234

→

-m - Specifies the model path to serve

→

-p - Sets the port number for the model server

Sends a prediction request to the served model and receives an explainability report along with the prediction.

Terminal

curl -X POST http://127.0.0.1:1234/invocations -H 'Content-Type: application/json' -d '{"data": [[5.1, 3.5, 1.4, 0.2]]}'

Expected Output

{"predictions": [0], "explanations": {"feature_importance": [0.8, 0.1, 0.05, 0.05]}}

Key Concept

If you remember nothing else from responsible AI, remember: always track, explain, and monitor your AI models to ensure fairness and trust.

Code Example

MLOps

import mlflow
from mlflow.models.signature import infer_signature
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
import pandas as pd

# Load data
iris = load_iris()
X = pd.DataFrame(iris.data, columns=iris.feature_names)
y = iris.target

# Train model
model = LogisticRegression(max_iter=200)
model.fit(X, y)

# Log model with MLflow including signature for input validation
signature = infer_signature(X, model.predict(X))
with mlflow.start_run() as run:
    mlflow.sklearn.log_model(model, "model", signature=signature)
    mlflow.log_param("max_iter", 200)
    mlflow.log_metric("accuracy", model.score(X, y))
print(f"Model logged in run {run.info.run_id}")

Common Mistakes

Ignoring fairness checks during model training

This can lead to biased models that harm certain groups or users.

Include fairness metrics and tests as part of your model training and evaluation pipeline.

Not enabling explainability features when serving models

Users and stakeholders cannot understand or trust AI decisions without explanations.

Serve models with explainability tools that provide feature importance or decision reasons.

Failing to monitor models after deployment

Models can degrade or become biased over time without detection.

Set up continuous monitoring and alerting for model performance and fairness.

Summary

Run MLflow projects that include responsible AI checks during model training.

Use MLflow UI to visualize model fairness, metrics, and explanations.

Serve models with explainability enabled for trustworthy predictions.

Send prediction requests and receive explanations to understand AI decisions.

Practice

1. What is the main goal of Responsible AI practices?

easy

A. To ensure AI systems are fair, safe, and trustworthy

B. To make AI run faster on all devices

C. To increase the complexity of AI models

D. To reduce the cost of AI hardware

Solution

1. Step 1: Understand the purpose of Responsible AI

   Responsible AI focuses on ethical and safe AI use, not speed or cost.

2. Step 2: Identify the key goals

   Fairness, safety, and trustworthiness are the core goals of Responsible AI.

3. Final Answer:

   To ensure AI systems are fair, safe, and trustworthy -> Option A

4. Quick Check:

   Responsible AI = fairness, safety, trust [OK]

Hint: Responsible AI means fairness and safety first [OK]

Common Mistakes:

• Confusing performance optimization with ethical goals
• Thinking cost reduction is the main focus
• Assuming complexity equals responsibility

2. Which of the following is a correct practice to check AI bias in a model?

easy

A. Using fairness metrics to evaluate model outputs

B. Avoiding transparency in model decisions

C. Only testing the model on training data

D. Ignoring data diversity during training

Solution

1. Step 1: Identify bias checking methods

   Bias checks require measuring fairness, not ignoring data or hiding decisions.

2. Step 2: Match correct practice

   Using fairness metrics helps detect bias in model outputs effectively.

3. Final Answer:

   Using fairness metrics to evaluate model outputs -> Option A

4. Quick Check:

   Bias check = fairness metrics [OK]

Hint: Use fairness metrics to spot bias [OK]

Common Mistakes:

• Ignoring diverse data leads to hidden bias
• Testing only on training data misses real bias
• Lack of transparency hides bias issues

3. Consider this Python snippet for monitoring AI model fairness:

fairness_scores = {'groupA': 0.85, 'groupB': 0.65}
if min(fairness_scores.values()) < 0.7:
    alert = 'Bias detected'
else:
    alert = 'Fair model'

What will be the value of alert after running this code?

medium

A. 'Fair model'

B. KeyError

C. TypeError

D. 'Bias detected'

Solution

1. Step 1: Evaluate fairness scores

   Values are 0.85 and 0.65; minimum is 0.65.

2. Step 2: Check condition in if statement

   Since 0.65 < 0.7, condition is true, so alert is set to 'Bias detected'.

3. Final Answer:

   'Bias detected' -> Option D

4. Quick Check:

   Min fairness < 0.7 means bias alert [OK]

Hint: Check minimum fairness score for bias alert [OK]

Common Mistakes:

• Confusing greater than and less than signs
• Expecting error instead of string output
• Ignoring dictionary value extraction

4. You have this code snippet to log AI model decisions for explainability:

def log_decision(input, decision):
    print(f"Input: {input}, Decision: {decision}")

log_decision('data1', decision)

What is the error in this code?

medium

A. Print statement syntax error

B. Function name is invalid

C. Missing quotes around 'decision' in function call

D. No error, code runs fine

Solution

1. Step 1: Check function call parameters

   The call uses decision without quotes, but decision is not defined as a variable.

2. Step 2: Identify correct usage

   To pass the string 'decision', it must be in quotes: 'decision'.

3. Final Answer:

   Missing quotes around 'decision' in function call -> Option C

4. Quick Check:

   Undefined variable needs quotes [OK]

Hint: Strings need quotes in function calls [OK]

Common Mistakes:

• Assuming variable 'decision' is predefined
• Ignoring syntax of print with f-string
• Thinking function name causes error

5. You want to build a monitoring system that alerts when AI model fairness drops below 0.75 and also logs explanations for decisions. Which combination of practices best supports Responsible AI?

hard

A. Only monitor model speed and ignore fairness

B. Use fairness metrics for alerts and log decision explanations transparently

C. Log decisions but do not monitor fairness scores

D. Monitor fairness but keep decision logic secret

Solution

1. Step 1: Identify key Responsible AI practices

   Responsible AI requires fairness monitoring and transparent explanations.

2. Step 2: Evaluate options for best fit

   Use fairness metrics for alerts and log decision explanations transparently combines fairness alerts and transparent logging, matching Responsible AI goals.

3. Final Answer:

   Use fairness metrics for alerts and log decision explanations transparently -> Option B

4. Quick Check:

   Fairness + transparency = Responsible AI [OK]

Hint: Combine fairness alerts with transparent logs [OK]

Common Mistakes:

• Ignoring fairness monitoring
• Hiding decision explanations
• Focusing only on performance metrics