Audit trails for model decisions in MLOps - Step-by-Step Execution

Process Flow - Audit trails for model decisions

Model receives input data

↓

Model makes decision

↓

Log decision details

↓

Store logs in audit trail system

↓

Audit trail available for review

↓

Use audit trail for debugging, compliance, or improvement

This flow shows how a model decision is logged step-by-step to create an audit trail for later review and compliance.

Execution Sample

MLOps

input_data = {'age': 30, 'income': 50000}
decision = model.predict(input_data)
audit_log = {
  'input': input_data,
  'decision': decision,
  'timestamp': current_time()
}
store_audit_log(audit_log)

This code logs the input, model decision, and timestamp to an audit trail system.

Process Table

Step	Action	Data Captured	Storage Location	Result
1	Receive input data	{'age': 30, 'income': 50000}	In memory	Input ready for prediction
2	Model makes decision	Prediction result (e.g., 'approve')	In memory	Decision generated
3	Create audit log entry	{input, decision, timestamp}	In memory object	Audit log prepared
4	Store audit log	Audit log object	Audit trail database or file	Audit log saved
5	Audit trail available	Stored logs	Audit trail system	Logs ready for review

💡 Audit trail is complete after storing the log for the model decision

Status Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	Final
input_data	None	{'age': 30, 'income': 50000}	{'age': 30, 'income': 50000}	{'age': 30, 'income': 50000}	{'age': 30, 'income': 50000}
decision	None	None	'approve'	'approve'	'approve'
audit_log	None	None	None	{'input': {'age': 30, 'income': 50000}, 'decision': 'approve', 'timestamp': '2024-06-01T12:00:00Z'}	{'input': {'age': 30, 'income': 50000}, 'decision': 'approve', 'timestamp': '2024-06-01T12:00:00Z'}

Key Moments - 3 Insights

Why do we log both input data and model decision in the audit trail?

Logging both input and decision ensures we can trace exactly what data led to each decision, as shown in execution_table rows 1 and 2.

What happens if the audit log is not stored properly after creation?

If the audit log is not saved (execution_table step 4), the audit trail is incomplete and cannot be reviewed later, losing traceability.

Why is a timestamp included in the audit log?

Timestamp records when the decision was made, helping order events and support compliance, as seen in the audit_log variable at step 3.

Visual Quiz - 3 Questions

Look at the execution_table at step 2. What data is captured?

A Input data only

B Prediction result only

C Input data and prediction result

D Timestamp only

💡 Hint

Check the 'Data Captured' column at step 2 in the execution_table.

At which step is the audit log saved to permanent storage?

A Step 4

B Step 3

C Step 1

D Step 5

💡 Hint

Look for the step where 'Store audit log' action happens in the execution_table.

If the input_data changes, which variable_tracker column will show the difference?

A After Step 2

B After Step 3

C After Step 1

D Final

💡 Hint

Check when input_data is first assigned in variable_tracker.

Concept Snapshot

Audit trails log model inputs, decisions, and timestamps.
Store logs in a secure system for review.
Helps with debugging, compliance, and trust.
Always capture input, output, and time.
Ensure logs are saved reliably.

Full Transcript

Audit trails for model decisions capture the input data, the model's decision, and the time the decision was made. This information is stored in a secure audit trail system. The process starts when the model receives input data, makes a decision, then creates a log entry with all details. This log is saved for later review to help with debugging, compliance, and improving the model. Key variables like input_data, decision, and audit_log change step-by-step as the process runs. Understanding when and what is logged helps ensure traceability and trust in automated decisions.