Automated testing for ML code in MLOps - Mini Project: Build & Apply

Automated Testing for ML Code

📖 Scenario: You are working on a machine learning project. To keep your code reliable, you want to add automated tests that check if your ML functions work correctly.This helps catch mistakes early, just like checking your homework before submitting it.

🎯 Goal: Build a simple automated test for a machine learning function that calculates accuracy.You will create test data, set expected results, write the test logic, and print the test outcome.

📋 What You'll Learn

Create a list of true labels and predicted labels

Set an expected accuracy value

Write a function to calculate accuracy

Write a test that compares calculated accuracy to expected accuracy

Print the test result as 'Test passed' or 'Test failed'

💡 Why This Matters

🌍 Real World

Automated tests help catch errors in ML code early, saving time and improving reliability before deploying models.

💼 Career

ML engineers and MLOps specialists use automated testing to ensure their models and code work correctly in production environments.

1

Create test data for labels

Create two lists called true_labels and predicted_labels with these exact values: true_labels = [1, 0, 1, 1, 0] and predicted_labels = [1, 0, 0, 1, 0].

MLOps

# Create the true_labels and predicted_labels lists
# Your code here

Hint

Use square brackets to create lists with the exact numbers given.

2

Set expected accuracy value

Create a variable called expected_accuracy and set it to the float value 0.8.

MLOps

true_labels = [1, 0, 1, 1, 0]
predicted_labels = [1, 0, 0, 1, 0]
# Set expected_accuracy to 0.8
# Your code here

Hint

Use a variable name exactly as expected_accuracy and assign it the value 0.8.

3

Write accuracy calculation function

Define a function called calculate_accuracy that takes two parameters: true_labels and predicted_labels. Inside, calculate accuracy as the number of matching labels divided by total labels, and return the accuracy as a float.

MLOps

true_labels = [1, 0, 1, 1, 0]
predicted_labels = [1, 0, 0, 1, 0]
expected_accuracy = 0.8
# Define calculate_accuracy function below
# Your code here

Hint

Use a for loop with zip to compare pairs of labels.

4

Write test and print result

Call calculate_accuracy(true_labels, predicted_labels) and store the result in actual_accuracy. Then write an if statement that prints 'Test passed' if actual_accuracy equals expected_accuracy, otherwise print 'Test failed'.

MLOps

true_labels = [1, 0, 1, 1, 0]
predicted_labels = [1, 0, 0, 1, 0]
expected_accuracy = 0.8
def calculate_accuracy(true_labels, predicted_labels):
    correct = 0
    for t, p in zip(true_labels, predicted_labels):
        if t == p:
            correct += 1
    return correct / len(true_labels)

# Call the function and print test result
# Your code here

Hint

Use an if statement to compare actual_accuracy and expected_accuracy.

Practice

1. What is the main purpose of automated testing in ML code?

easy

A. To replace the need for data cleaning

B. To make the code run faster

C. To increase the size of the dataset

D. To catch bugs early and keep the code reliable

Solution

1. Step 1: Understand the role of automated testing

   Automated testing is used to check if code works correctly without manual checks.

2. Step 2: Identify the main benefit in ML context

   In ML, it helps find bugs early and keeps the code reliable during changes.

3. Final Answer:

   To catch bugs early and keep the code reliable -> Option D

4. Quick Check:

   Automated testing = catch bugs early [OK]

Hint: Automated tests find bugs early to keep code safe [OK]

Common Mistakes:

• Thinking automated tests speed up code
• Confusing testing with data processing
• Believing tests replace data cleaning

2. Which of the following is the correct way to write a simple test function in Python for ML code?

easy

A. test_accuracy: assert model_accuracy > 0.8

B. def test_accuracy(): assert model_accuracy > 0.8

C. function test_accuracy() { assert model_accuracy > 0.8 }

D. def test_accuracy: assert model_accuracy > 0.8

Solution

1. Step 1: Recognize Python test function syntax

   In Python, functions start with 'def' and have parentheses and a colon.

2. Step 2: Check each option's syntax

   def test_accuracy(): assert model_accuracy > 0.8 uses correct Python syntax with 'def' and parentheses. Others have syntax errors or wrong language style.

3. Final Answer:

   def test_accuracy(): assert model_accuracy > 0.8 -> Option B

4. Quick Check:

   Python test function = def + parentheses + colon [OK]

Hint: Python functions need def, parentheses, and colon [OK]

Common Mistakes:

• Omitting parentheses in function definition
• Using JavaScript syntax in Python
• Missing colon after function header

3. Given the test function below, what will be the output when running it if model_accuracy = 0.75?

def test_accuracy():
    assert model_accuracy > 0.8, "Accuracy too low"

test_accuracy()

medium

A. AssertionError: Accuracy too low

B. TypeError

C. SyntaxError

D. No output, test passes

Solution

1. Step 1: Understand the assert statement

   The assert checks if model_accuracy > 0.8. If false, it raises AssertionError with message.

2. Step 2: Evaluate the condition with model_accuracy = 0.75

   0.75 is not greater than 0.8, so assertion fails and raises error with message "Accuracy too low".

3. Final Answer:

   AssertionError: Accuracy too low -> Option A

4. Quick Check:

   Assert false triggers AssertionError [OK]

Hint: Assert fails if condition false, shows error message [OK]

Common Mistakes:

• Thinking assert prints message on success
• Confusing AssertionError with SyntaxError
• Ignoring the error message text

4. You wrote this test function but it raises a SyntaxError. What is the mistake?

def test_model():
    assert model.predict(X) == y
    print("Test passed")

 test_model()

medium

A. Indentation error before calling test_model()

B. Missing colon after function definition

C. assert statement syntax is wrong

D. print statement is not allowed in tests

Solution

1. Step 1: Check indentation of function call

   The call to test_model() is indented, which is invalid outside function or block.

2. Step 2: Confirm other syntax parts are correct

   Function definition has colon, assert syntax is correct, print is allowed. Only indentation is wrong.

3. Final Answer:

   Indentation error before calling test_model() -> Option A

4. Quick Check:

   Top-level calls must not be indented [OK]

Hint: Top-level code must not be indented [OK]

Common Mistakes:

• Indenting function calls at top level
• Confusing assert syntax errors
• Thinking print is disallowed in tests

5. You want to automate testing for your ML model training function that returns accuracy. Which approach best ensures your tests catch unexpected accuracy drops?

hard

A. Write tests that print accuracy without checking values

B. Write tests that only check if training runs without errors

C. Write tests that assert accuracy is above a set threshold after training

D. Write tests that compare accuracy to previous run without threshold

Solution

1. Step 1: Identify goal of testing accuracy

   We want to detect if accuracy drops unexpectedly, so tests must check accuracy value.

2. Step 2: Evaluate options for effectiveness

   Write tests that assert accuracy is above a set threshold after training asserts accuracy above threshold, catching drops. Write tests that print accuracy without checking values only prints, no check. Write tests that only check if training runs without errors ignores accuracy value. Write tests that compare accuracy to previous run without threshold compares to previous run but no threshold, may miss small drops.

3. Final Answer:

   Write tests that assert accuracy is above a set threshold after training -> Option C

4. Quick Check:

   Assert accuracy > threshold catches drops [OK]

Hint: Assert accuracy above threshold to catch drops [OK]

Common Mistakes:

• Not asserting accuracy value
• Only printing results without checks
• Ignoring accuracy thresholds in tests