Automated testing for ML code in MLOps - Commands & Configuration

Introduction

Automated testing for ML code helps catch errors early by running checks on your machine learning scripts automatically. It ensures your ML models and data processing work as expected without manual testing every time you change code.

When you want to verify that your data preprocessing functions handle different input formats correctly

When you need to check that your model training code produces consistent results after changes

When you want to ensure your evaluation metrics calculations are accurate

When you want to catch bugs in feature engineering before deploying models

When you want to automate testing to save time and avoid manual errors

Commands

Run automated tests defined in the test_ml_code.py file to check ML code correctness.

Terminal

pytest test_ml_code.py

Expected Output

============================= test session starts ==============================

collected 3 items



test_ml_code.py ...                                                      [100%]



============================== 3 passed in 0.05s ===============================

→

-v - Show detailed test results

→

-k - Run tests matching a keyword expression

Discover and run all unit tests in the 'tests' directory to validate ML code components.

Terminal

python -m unittest discover tests

Expected Output

...

----------------------------------------------------------------------

Ran 3 tests in 0.012s



OK

→

-v - Run tests with verbose output

Key Concept

If you remember nothing else from this pattern, remember: automated tests catch ML code errors early and save time by running checks automatically.

Code Example

MLOps

import unittest
import numpy as np

def preprocess(data):
    return (data - np.mean(data)) / np.std(data)

class TestMLCode(unittest.TestCase):
    def test_preprocess(self):
        data = np.array([1, 2, 3, 4, 5])
        processed = preprocess(data)
        self.assertAlmostEqual(np.mean(processed), 0, places=6)
        self.assertAlmostEqual(np.std(processed), 1, places=6)

    def test_model_training(self):
        np.random.seed(0)
        data = np.random.rand(10, 2)
        labels = np.random.randint(0, 2, 10)
        # Dummy training: sum features > 1 predicts 1 else 0
        predictions = (np.sum(data, axis=1) > 1).astype(int)
        accuracy = np.mean(predictions == labels)
        self.assertGreaterEqual(accuracy, 0)

    def test_metric_calculation(self):
        true = np.array([1, 0, 1, 1])
        pred = np.array([1, 0, 0, 1])
        accuracy = np.mean(true == pred)
        self.assertEqual(accuracy, 0.75)

if __name__ == '__main__':
    unittest.main()

Common Mistakes

Not writing tests for data preprocessing functions

Data issues can silently cause model failures or poor performance if preprocessing is not tested.

Write tests that check data inputs and outputs for preprocessing steps.

Ignoring test failures and continuing development

Ignoring failures can let bugs accumulate and cause bigger problems later.

Fix test failures immediately to keep code reliable.

Writing tests that depend on random outputs without fixing seeds

Tests may fail unpredictably due to randomness in ML code.

Set random seeds in tests to ensure consistent results.

Summary

Use pytest or unittest commands to run automated tests on ML code.

Write tests for data preprocessing, model training, and metric calculations.

Fix test failures immediately to keep ML code reliable and consistent.

Practice

1. What is the main purpose of automated testing in ML code?

easy

A. To replace the need for data cleaning

B. To make the code run faster

C. To increase the size of the dataset

D. To catch bugs early and keep the code reliable

Solution

1. Step 1: Understand the role of automated testing

   Automated testing is used to check if code works correctly without manual checks.

2. Step 2: Identify the main benefit in ML context

   In ML, it helps find bugs early and keeps the code reliable during changes.

3. Final Answer:

   To catch bugs early and keep the code reliable -> Option D

4. Quick Check:

   Automated testing = catch bugs early [OK]

Hint: Automated tests find bugs early to keep code safe [OK]

Common Mistakes:

• Thinking automated tests speed up code
• Confusing testing with data processing
• Believing tests replace data cleaning

2. Which of the following is the correct way to write a simple test function in Python for ML code?

easy

A. test_accuracy: assert model_accuracy > 0.8

B. def test_accuracy(): assert model_accuracy > 0.8

C. function test_accuracy() { assert model_accuracy > 0.8 }

D. def test_accuracy: assert model_accuracy > 0.8

Solution

1. Step 1: Recognize Python test function syntax

   In Python, functions start with 'def' and have parentheses and a colon.

2. Step 2: Check each option's syntax

   def test_accuracy(): assert model_accuracy > 0.8 uses correct Python syntax with 'def' and parentheses. Others have syntax errors or wrong language style.

3. Final Answer:

   def test_accuracy(): assert model_accuracy > 0.8 -> Option B

4. Quick Check:

   Python test function = def + parentheses + colon [OK]

Hint: Python functions need def, parentheses, and colon [OK]

Common Mistakes:

• Omitting parentheses in function definition
• Using JavaScript syntax in Python
• Missing colon after function header

3. Given the test function below, what will be the output when running it if model_accuracy = 0.75?

def test_accuracy():
    assert model_accuracy > 0.8, "Accuracy too low"

test_accuracy()

medium

A. AssertionError: Accuracy too low

B. TypeError

C. SyntaxError

D. No output, test passes

Solution

1. Step 1: Understand the assert statement

   The assert checks if model_accuracy > 0.8. If false, it raises AssertionError with message.

2. Step 2: Evaluate the condition with model_accuracy = 0.75

   0.75 is not greater than 0.8, so assertion fails and raises error with message "Accuracy too low".

3. Final Answer:

   AssertionError: Accuracy too low -> Option A

4. Quick Check:

   Assert false triggers AssertionError [OK]

Hint: Assert fails if condition false, shows error message [OK]

Common Mistakes:

• Thinking assert prints message on success
• Confusing AssertionError with SyntaxError
• Ignoring the error message text

4. You wrote this test function but it raises a SyntaxError. What is the mistake?

def test_model():
    assert model.predict(X) == y
    print("Test passed")

 test_model()

medium

A. Indentation error before calling test_model()

B. Missing colon after function definition

C. assert statement syntax is wrong

D. print statement is not allowed in tests

Solution

1. Step 1: Check indentation of function call

   The call to test_model() is indented, which is invalid outside function or block.

2. Step 2: Confirm other syntax parts are correct

   Function definition has colon, assert syntax is correct, print is allowed. Only indentation is wrong.

3. Final Answer:

   Indentation error before calling test_model() -> Option A

4. Quick Check:

   Top-level calls must not be indented [OK]

Hint: Top-level code must not be indented [OK]

Common Mistakes:

• Indenting function calls at top level
• Confusing assert syntax errors
• Thinking print is disallowed in tests

5. You want to automate testing for your ML model training function that returns accuracy. Which approach best ensures your tests catch unexpected accuracy drops?

hard

A. Write tests that print accuracy without checking values

B. Write tests that only check if training runs without errors

C. Write tests that assert accuracy is above a set threshold after training

D. Write tests that compare accuracy to previous run without threshold

Solution

1. Step 1: Identify goal of testing accuracy

   We want to detect if accuracy drops unexpectedly, so tests must check accuracy value.

2. Step 2: Evaluate options for effectiveness

   Write tests that assert accuracy is above a set threshold after training asserts accuracy above threshold, catching drops. Write tests that print accuracy without checking values only prints, no check. Write tests that only check if training runs without errors ignores accuracy value. Write tests that compare accuracy to previous run without threshold compares to previous run but no threshold, may miss small drops.

3. Final Answer:

   Write tests that assert accuracy is above a set threshold after training -> Option C

4. Quick Check:

   Assert accuracy > threshold catches drops [OK]

Hint: Assert accuracy above threshold to catch drops [OK]

Common Mistakes:

• Not asserting accuracy value
• Only printing results without checks
• Ignoring accuracy thresholds in tests