Audit trails for model decisions in MLOps - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the primary purpose of audit trails in model decisions?

Audit trails are used in machine learning operations (MLOps) to track model decisions. What is the main reason for keeping these audit trails?

A To automatically improve model accuracy without human intervention

B To speed up the model training process by caching previous results

C To encrypt the model parameters for security reasons

D To record the sequence of steps and data that led to each model decision for transparency and debugging

💡 Hint

Think about why you might want to review or explain a model's output later.

Audit trails provide a clear record of how a model arrived at a decision. This helps with transparency, debugging, and compliance.

💻 Command Output

intermediate

What is the output of this command logging a model decision?

Given the following logging command in a model inference pipeline, what will be the output in the audit log?

MLOps

import json

model_decision = {'input_id': 123, 'prediction': 'approved', 'confidence': 0.92}
log_entry = json.dumps(model_decision)
print(log_entry)

A {input_id: 123, prediction: approved, confidence: 0.92}

B ['input_id', 123, 'prediction', 'approved', 'confidence', 0.92]

C {"input_id": 123, "prediction": "approved", "confidence": 0.92}

D ('input_id', 123, 'prediction', 'approved', 'confidence', 0.92)

💡 Hint

Remember how Python's json.dumps formats dictionaries as strings.

json.dumps converts a Python dictionary into a JSON-formatted string with keys and string values quoted properly.

❓ Configuration

advanced

Which configuration snippet correctly enables audit logging in a model deployment YAML?

You want to enable audit logging for model decisions in your deployment configuration. Which YAML snippet correctly sets this up?

A

audit:
  enabled: true
  log_path: /var/log/model_audit.log
  log_level: info

B

audit:
  enable: yes
  path: /var/log/model_audit.log
  level: info

C

logging:
  audit_enabled: true
  audit_log: /var/log/model_audit.log
  level: info

D

audit_log:
  active: true
  file: /var/log/model_audit.log
  severity: info

💡 Hint

Look for correct YAML keys and boolean values.

Option A uses correct YAML keys and boolean values to enable audit logging properly.

❓ Troubleshoot

advanced

Why does the audit trail not record model decisions despite logging enabled?

You enabled audit logging in your model deployment, but no audit entries appear in the log file. What is the most likely cause?

A The model inference code does not call the logging function after each decision

B The log file path is set correctly but the disk is full

C The audit logging is enabled but the model is not deployed

D The model decisions are encrypted and cannot be logged

💡 Hint

Think about what must happen in the code to create log entries.

Even if audit logging is enabled, if the code does not explicitly log each decision, no entries will appear.

🔀 Workflow

expert

What is the correct order of steps to implement audit trails for model decisions?

Arrange the following steps in the correct order to implement audit trails for model decisions in an MLOps pipeline.

A 2,1,3,4

B 1,2,3,4

C 1,3,2,4

D 3,1,2,4

💡 Hint

Think about planning first, then coding, then configuring storage, then monitoring.

First decide what to log, then add logging in code, then set up storage, and finally monitor logs.