Automated testing for ML code in MLOps - Time & Space Complexity

Time Complexity: Automated testing for ML code

O(n)

Understanding Time Complexity

When we run automated tests on machine learning code, we want to know how the time to run these tests changes as the code or data grows.

We ask: How does testing time increase when we add more tests or bigger data?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

for test_case in test_suite:
    model_output = model.predict(test_case.input_data)
    assert model_output == test_case.expected_output

This code runs each test case by making the model predict and then checking the result.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Looping through each test case in the test suite.
• How many times: Once per test case, so as many times as there are tests.

How Execution Grows With Input

As the number of test cases grows, the total time to run all tests grows roughly the same way.

Input Size (n)	Approx. Operations
10	10 model predictions and checks
100	100 model predictions and checks
1000	1000 model predictions and checks

Pattern observation: Doubling the number of tests roughly doubles the total work.

Final Time Complexity

Time Complexity: O(n)

This means the testing time grows directly in proportion to the number of test cases.

Common Mistake

[X] Wrong: "Running more tests won't affect total time much because each test is fast."

[OK] Correct: Even if each test is quick, many tests add up, so total time grows with the number of tests.

Interview Connect

Understanding how test time grows helps you plan testing strategies and shows you can think about code efficiency beyond just writing tests.

Self-Check

"What if each test case input data size also grows with n? How would the time complexity change then?"

Practice

1. What is the main purpose of automated testing in ML code?

easy

A. To replace the need for data cleaning

B. To make the code run faster

C. To increase the size of the dataset

D. To catch bugs early and keep the code reliable

Solution

1. Step 1: Understand the role of automated testing

   Automated testing is used to check if code works correctly without manual checks.

2. Step 2: Identify the main benefit in ML context

   In ML, it helps find bugs early and keeps the code reliable during changes.

3. Final Answer:

   To catch bugs early and keep the code reliable -> Option D

4. Quick Check:

   Automated testing = catch bugs early [OK]

Hint: Automated tests find bugs early to keep code safe [OK]

Common Mistakes:

• Thinking automated tests speed up code
• Confusing testing with data processing
• Believing tests replace data cleaning

2. Which of the following is the correct way to write a simple test function in Python for ML code?

easy

A. test_accuracy: assert model_accuracy > 0.8

B. def test_accuracy(): assert model_accuracy > 0.8

C. function test_accuracy() { assert model_accuracy > 0.8 }

D. def test_accuracy: assert model_accuracy > 0.8

Solution

1. Step 1: Recognize Python test function syntax

   In Python, functions start with 'def' and have parentheses and a colon.

2. Step 2: Check each option's syntax

   def test_accuracy(): assert model_accuracy > 0.8 uses correct Python syntax with 'def' and parentheses. Others have syntax errors or wrong language style.

3. Final Answer:

   def test_accuracy(): assert model_accuracy > 0.8 -> Option B

4. Quick Check:

   Python test function = def + parentheses + colon [OK]

Hint: Python functions need def, parentheses, and colon [OK]

Common Mistakes:

• Omitting parentheses in function definition
• Using JavaScript syntax in Python
• Missing colon after function header

3. Given the test function below, what will be the output when running it if model_accuracy = 0.75?

def test_accuracy():
    assert model_accuracy > 0.8, "Accuracy too low"

test_accuracy()

medium

A. AssertionError: Accuracy too low

B. TypeError

C. SyntaxError

D. No output, test passes

Solution

1. Step 1: Understand the assert statement

   The assert checks if model_accuracy > 0.8. If false, it raises AssertionError with message.

2. Step 2: Evaluate the condition with model_accuracy = 0.75

   0.75 is not greater than 0.8, so assertion fails and raises error with message "Accuracy too low".

3. Final Answer:

   AssertionError: Accuracy too low -> Option A

4. Quick Check:

   Assert false triggers AssertionError [OK]

Hint: Assert fails if condition false, shows error message [OK]

Common Mistakes:

• Thinking assert prints message on success
• Confusing AssertionError with SyntaxError
• Ignoring the error message text

4. You wrote this test function but it raises a SyntaxError. What is the mistake?

def test_model():
    assert model.predict(X) == y
    print("Test passed")

 test_model()

medium

A. Indentation error before calling test_model()

B. Missing colon after function definition

C. assert statement syntax is wrong

D. print statement is not allowed in tests

Solution

1. Step 1: Check indentation of function call

   The call to test_model() is indented, which is invalid outside function or block.

2. Step 2: Confirm other syntax parts are correct

   Function definition has colon, assert syntax is correct, print is allowed. Only indentation is wrong.

3. Final Answer:

   Indentation error before calling test_model() -> Option A

4. Quick Check:

   Top-level calls must not be indented [OK]

Hint: Top-level code must not be indented [OK]

Common Mistakes:

• Indenting function calls at top level
• Confusing assert syntax errors
• Thinking print is disallowed in tests

5. You want to automate testing for your ML model training function that returns accuracy. Which approach best ensures your tests catch unexpected accuracy drops?

hard

A. Write tests that print accuracy without checking values

B. Write tests that only check if training runs without errors

C. Write tests that assert accuracy is above a set threshold after training

D. Write tests that compare accuracy to previous run without threshold

Solution

1. Step 1: Identify goal of testing accuracy

   We want to detect if accuracy drops unexpectedly, so tests must check accuracy value.

2. Step 2: Evaluate options for effectiveness

   Write tests that assert accuracy is above a set threshold after training asserts accuracy above threshold, catching drops. Write tests that print accuracy without checking values only prints, no check. Write tests that only check if training runs without errors ignores accuracy value. Write tests that compare accuracy to previous run without threshold compares to previous run but no threshold, may miss small drops.

3. Final Answer:

   Write tests that assert accuracy is above a set threshold after training -> Option C

4. Quick Check:

   Assert accuracy > threshold catches drops [OK]

Hint: Assert accuracy above threshold to catch drops [OK]

Common Mistakes:

• Not asserting accuracy value
• Only printing results without checks
• Ignoring accuracy thresholds in tests