Audit trails for model decisions in MLOps - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to log model predictions to a file.

MLOps

with open('audit_log.txt', '[1]') as f:
    f.write(f"Prediction: {prediction}\n")

Drag options to blanks, or click blank then click option'

Ar

Ba

Cw+

Dx

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'w' which overwrites the file, losing previous logs.

Use 'a' to append logs without overwriting existing data.

2fill in blank

medium

Complete the code to include a timestamp in the audit log entry.

MLOps

import datetime

log_entry = f"{datetime.datetime.now()}: Prediction = {prediction}"
with open('audit_log.txt', 'a') as f:
    f.write([1] + '\n')

Drag options to blanks, or click blank then click option'

Alog_entry()

Bstr(log_entry)

Crepr(log_entry)

Dlog_entry

Attempts:

3 left

💡 Hint

Common Mistakes

Calling log_entry() which causes an error because it's not a function.

log_entry is already a string, so write it directly.

3fill in blank

hard

Fix the error in the code that logs model input and output as JSON.

MLOps

import json

log_data = {'input': model_input, 'output': model_output}
with open('audit_log.json', 'a') as f:
    f.write(json.dumps([1]) + '\n')

Drag options to blanks, or click blank then click option'

Astr(log_data)

Blog_data()

Clog_data

Djson.dumps(log_data)

Attempts:

3 left

💡 Hint

Common Mistakes

Passing log_data() which causes a TypeError.

json.dumps expects a dictionary, so pass log_data directly.

4fill in blank

hard

Fill both blanks to create a dictionary comprehension that logs only features with values above 0.5.

MLOps

filtered_log = {key: value for key, value in model_features.items() if value [1] 0.5 and key [2] 'confidence'}

Drag options to blanks, or click blank then click option'

A>

B<

C!=

D==

Attempts:

3 left

💡 Hint

Common Mistakes

Using '<' instead of '>' for value comparison.

Using '==' instead of '!=' for key filtering.

We want values greater than 0.5 and keys not equal to 'confidence'.

5fill in blank

hard

Fill all three blanks to create a filtered audit log dictionary with uppercase keys, values above 0.7, and keys not equal to 'bias'.

MLOps

filtered_audit = [1]: [2] for [3], val in audit_data.items() if val > 0.7 and [3] != 'bias'

Drag options to blanks, or click blank then click option'

Akey.upper()

Bval

Ckey

Dvalue

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'value' instead of 'val' for values.

Using 'value' or 'val' as loop variable instead of 'key'.

Use key.upper() for keys, val for values, and key as the loop variable.

Practice

1. What is the main purpose of audit trails in machine learning model decisions?

easy

A. To encrypt the model data for security

B. To speed up the model training process

C. To reduce the size of the model

D. To record inputs, outputs, and context for each model decision

Solution

1. Step 1: Understand audit trail purpose

   Audit trails are used to keep a record of what data was input, what output was produced, and the context around the decision.

2. Step 2: Compare options

   Only To record inputs, outputs, and context for each model decision describes this purpose correctly. Other options describe unrelated tasks.

3. Final Answer:

   To record inputs, outputs, and context for each model decision -> Option D

4. Quick Check:

   Audit trails = record inputs and outputs [OK]

Hint: Audit trails track what goes in and out of models [OK]

Common Mistakes:

• Confusing audit trails with model optimization
• Thinking audit trails speed up training
• Believing audit trails encrypt data

2. Which of the following is the correct way to log a model decision with timestamp in Python?

easy

A. log_entry = f"{datetime.now()} - Input: {input_data}, Output: {output}"

B. log_entry = datetime.now() + input_data + output

C. log_entry = "Input: input_data, Output: output"

D. log_entry = f"Input: {input_data} Output: {output}"

Solution

1. Step 1: Check correct string formatting with timestamp

   log_entry = f"{datetime.now()} - Input: {input_data}, Output: {output}" uses f-string with datetime.now() to include timestamp and variables properly.

2. Step 2: Identify errors in other options

   log_entry = datetime.now() + input_data + output tries to add incompatible types, causing error. Options C and D miss timestamp or variable interpolation.

3. Final Answer:

   log_entry = f"{datetime.now()} - Input: {input_data}, Output: {output}" -> Option A

4. Quick Check:

   Use f-string with datetime.now() for logging [OK]

Hint: Use f-strings and datetime.now() for timestamped logs [OK]

Common Mistakes:

• Concatenating incompatible types without conversion
• Forgetting to include timestamp
• Not using variable interpolation in strings

3. Given the following Python code snippet for logging model decisions, what will be the output?

from datetime import datetime
input_data = {'age': 30}
output = 'approved'
log_entry = f"{datetime(2024, 6, 1, 12, 0)} - Input: {input_data}, Output: {output}"
print(log_entry)

medium

A. 2024/06/01 12:00 - Input: {'age': 30}, Output: approved

B. datetime.datetime(2024, 6, 1, 12, 0) - Input: {'age': 30}, Output: approved

C. 2024-06-01 12:00:00 - Input: {'age': 30}, Output: approved

D. Error: datetime object cannot be formatted in f-string

Solution

1. Step 1: Understand datetime object formatting in f-string

   Using datetime(2024, 6, 1, 12, 0) in f-string calls its __str__ method, which outputs '2024-06-01 12:00:00'.

2. Step 2: Combine string parts

   The rest of the string includes input_data and output as expected, so the full string prints correctly.

3. Final Answer:

   2024-06-01 12:00:00 - Input: {'age': 30}, Output: approved -> Option C

4. Quick Check:

   Datetime __str__ = 'YYYY-MM-DD HH:MM:SS' [OK]

Hint: Datetime prints as 'YYYY-MM-DD HH:MM:SS' in f-strings [OK]

Common Mistakes:

• Expecting datetime object to print as constructor call
• Confusing date formats
• Thinking f-string cannot handle datetime objects

4. You have this code snippet to log model decisions but it raises an error:

log_entry = f"{datetime.now()} - Input: {input_data}, Output: {output}"

What is the most likely cause of the error?

medium

A. datetime module is not imported

B. input_data is not defined

C. f-string syntax is incorrect

D. output variable is a number, not a string

Solution

1. Step 1: Check for datetime usage

   Using datetime.now() requires importing datetime module or class. If missing, NameError occurs.

2. Step 2: Verify other variables and syntax

   input_data and output can be any type; f-string handles them. Syntax is correct.

3. Final Answer:

   datetime module is not imported -> Option A

4. Quick Check:

   Missing import datetime causes NameError [OK]

Hint: Always import datetime before using datetime.now() [OK]

Common Mistakes:

• Assuming variables cause error without checking imports
• Thinking f-string syntax is wrong
• Believing numbers cause f-string errors

5. You want to create an audit trail that records model version, input data, output, and timestamp in JSON format for each decision. Which Python code snippet correctly creates this audit trail entry?

hard

A. import json, datetime
audit_entry = json.dumps({"model_version": "v1.2", "input": input_data, "output": output, "timestamp": datetime.datetime.now.isoformat()})

B. import json, datetime
audit_entry = json.dumps({"model_version": "v1.2", "input": input_data, "output": output, "timestamp": datetime.now().isoformat()})

C. import json, datetime
audit_entry = json.dumps({"model_version": "v1.2", "input": input_data, "output": output, "timestamp": datetime.now().str()})

D. import json, datetime
audit_entry = json.dumps({"model_version": "v1.2", "input": input_data, "output": output, "timestamp": datetime.now()})

Solution

1. Step 1: Check correct import and datetime usage

   import json, datetime audit_entry = json.dumps({"model_version": "v1.2", "input": input_data, "output": output, "timestamp": datetime.now().isoformat()}) correctly imports datetime and uses datetime.now().isoformat() to get a string timestamp.

2. Step 2: Validate JSON serialization

   datetime.now() returns a datetime object which is not JSON serializable directly, so isoformat() converts it to string. import json, datetime audit_entry = json.dumps({"model_version": "v1.2", "input": input_data, "output": output, "timestamp": datetime.now()}) fails here.

3. Step 3: Check other options

   import json, datetime audit_entry = json.dumps({"model_version": "v1.2", "input": input_data, "output": output, "timestamp": datetime.datetime.now.isoformat()}) tries to call isoformat on the now method object (missing () after now), causing AttributeError. import json, datetime audit_entry = json.dumps({"model_version": "v1.2", "input": input_data, "output": output, "timestamp": datetime.now().str()}) tries to call .str() on datetime object, causing AttributeError.

4. Final Answer:

   import json, datetime audit_entry = json.dumps({"model_version": "v1.2", "input": input_data, "output": output, "timestamp": datetime.now().isoformat()}) -> Option B

5. Quick Check:

   Use datetime.now().isoformat() for JSON timestamp [OK]

Hint: Use datetime.now().isoformat() for JSON-friendly timestamps [OK]

Common Mistakes:

• Missing () after now() leading to method object error
• Trying to serialize datetime object directly
• Using non-existent .str() method on datetime