Regression testing in Testing Fundamentals - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Purpose of Regression Testing in Data Science Projects

What is the main purpose of regression testing when applied to data science models?

A To check if new code changes have broken existing model functionality

B To improve the accuracy of the model by retraining with new data

C To visualize the model predictions over time

D To optimize the hyperparameters of the model

💡 Hint

Think about what regression testing means in software and how it applies to models.

Regression testing ensures that recent changes do not break or degrade existing functionality, including model predictions.

❓ Predict Output

intermediate

Output of Regression Test on Model Predictions

Given the following Python code that compares old and new model predictions, what is the output?

Testing Fundamentals

old_preds = [0.8, 0.6, 0.9, 0.4]
new_preds = [0.8, 0.65, 0.9, 0.4]
threshold = 0.05
changes = [abs(o - n) for o, n in zip(old_preds, new_preds)]
regression_fail = any(change > threshold for change in changes)
print(regression_fail)

A True

B TypeError

C SyntaxError

D False

💡 Hint

Check if any prediction difference exceeds the threshold.

The differences are 0.0, 0.05, 0.0, 0.0. None exceed 0.05, so regression_fail is False.

❓ data_output

advanced

Detecting Regression Failures in Model Metrics

You have two sets of model evaluation metrics stored in dictionaries. Which option correctly identifies if any metric has regressed (got worse)?

Testing Fundamentals

old_metrics = {'accuracy': 0.92, 'precision': 0.85, 'recall': 0.88}
new_metrics = {'accuracy': 0.90, 'precision': 0.87, 'recall': 0.85}
regressions = {k: new_metrics[k] < old_metrics[k] for k in old_metrics}
print(regressions)

A {'accuracy': True, 'precision': True, 'recall': True}

B {'accuracy': False, 'precision': True, 'recall': False}

C {'accuracy': True, 'precision': False, 'recall': True}

D {'accuracy': False, 'precision': False, 'recall': False}

💡 Hint

Compare each metric's new value to the old value to see if it decreased.

Accuracy dropped from 0.92 to 0.90 (True), precision increased (False), recall dropped (True).

🔧 Debug

advanced

Identify the Error in Regression Test Code

What error does the following regression test code raise?

Testing Fundamentals

old_results = [0.7, 0.8, 0.9]
new_results = [0.7, 0.85]
for i in range(len(old_results)):
    if abs(old_results[i] - new_results[i]) > 0.1:
        print('Regression detected')

A IndexError

B TypeError

C No error, prints 'Regression detected'

D SyntaxError

💡 Hint

Check if the loop index matches the length of both lists.

new_results has fewer elements than old_results, so accessing new_results[2] causes IndexError.

🚀 Application

expert

Choosing the Best Regression Testing Strategy for Model Updates

You update a machine learning model frequently with new data. Which regression testing strategy best ensures that model performance does not degrade over time?

A Only retrain the model without any testing since new data should improve performance

B Run automated tests comparing new model predictions to a fixed baseline dataset and alert if differences exceed a threshold

C Manually check model outputs occasionally after major code changes

D Use random sampling of predictions without comparing to previous results

💡 Hint

Think about automation and consistent checks for regression.

Automated tests comparing predictions to a baseline help catch unintended degradations quickly and reliably.

Practice

1.

What is the main purpose of regression testing in data science projects?

easy

A. To check that new changes do not break existing results

B. To create new models from scratch

C. To collect new data for training

D. To visualize data trends

Solution

1. Step 1: Understand regression testing goal

   Regression testing is about verifying that recent changes do not cause errors in existing functionality.

2. Step 2: Match purpose with options

   Only To check that new changes do not break existing results describes this goal correctly, others describe unrelated tasks.

3. Final Answer:

   To check that new changes do not break existing results -> Option A

4. Quick Check:

   Regression testing = check no breakage [OK]

Hint: Regression testing ensures old results still work after changes [OK]

Common Mistakes:

• Confusing regression testing with model building
• Thinking it collects new data
• Assuming it is for data visualization

2.

Which of the following is the correct way to describe regression testing?

Regression testing:

def regression_test(old_output, new_output):
    return old_output == new_output

easy

A. It compares old and new outputs to detect changes

B. It trains a new model using old and new data

C. It visualizes differences between datasets

D. It deletes old test results before running new tests

Solution

1. Step 1: Analyze the function code

   The function returns True if old_output equals new_output, meaning it compares outputs.

2. Step 2: Match function behavior to options

   It compares old and new outputs to detect changes correctly states it compares outputs to detect changes; others describe unrelated actions.

3. Final Answer:

   It compares old and new outputs to detect changes -> Option A

4. Quick Check:

   Function compares outputs = It compares old and new outputs to detect changes [OK]

Hint: Regression test checks if old and new outputs match exactly [OK]

Common Mistakes:

• Thinking regression testing retrains models
• Assuming it creates visualizations
• Believing it deletes old results

3.

Given the following Python code for regression testing outputs, what will be the printed result?

old_results = [10, 20, 30]
new_results = [10, 20, 25]

if old_results == new_results:
    print("Test Passed")
else:
    print("Test Failed")

medium

A. Test Passed

B. SyntaxError

C. Test Failed

D. No output

Solution

1. Step 1: Compare the two lists element-wise

   old_results has [10, 20, 30], new_results has [10, 20, 25]. The last elements differ (30 vs 25).

2. Step 2: Evaluate the if condition

   Since the lists are not equal, the else block runs, printing "Test Failed".

3. Final Answer:

   Test Failed -> Option C

4. Quick Check:

   Lists differ, so output = Test Failed [OK]

Hint: Lists must match exactly for test to pass [OK]

Common Mistakes:

• Assuming partial match passes test
• Confusing list equality with set equality
• Expecting syntax errors from correct code

4.

Identify the error in this regression testing code snippet:

def regression_test(old, new):
    if old = new:
        return True
    else:
        return False

medium

A. Missing return statement

B. Using assignment operator '=' instead of comparison '==' in if condition

C. Function should print results, not return

D. Incorrect function name

Solution

1. Step 1: Check the if condition syntax

   The condition uses '=' which is assignment, not comparison. Python requires '==' for comparison.

2. Step 2: Identify correct syntax for comparison

   Replacing '=' with '==' fixes the syntax error and allows proper comparison.

3. Final Answer:

   Using assignment operator '=' instead of comparison '==' in if condition -> Option B

4. Quick Check:

   Use '==' to compare, not '=' [OK]

Hint: Use '==' for comparison, '=' is assignment [OK]

Common Mistakes:

• Confusing '=' and '==' in conditions
• Thinking return is missing
• Believing function name matters for syntax

5.

You have a data science model that outputs predictions as a list. You want to automate regression testing to check if new code changes break the model's output. Which approach below best applies regression testing?

hard

A. Delete old predictions and start fresh after every change

B. Train a new model with updated data and ignore old predictions

C. Visualize new predictions only to check for differences

D. Save previous predictions and compare them with new predictions after changes

Solution

1. Step 1: Understand regression testing in model outputs

   Regression testing means re-running tests and comparing new outputs to old outputs to detect unintended changes.

2. Step 2: Evaluate options for best fit

   Save previous predictions and compare them with new predictions after changes saves old predictions and compares them to new ones, matching regression testing. Others ignore old outputs or delete them, which is not regression testing.

3. Final Answer:

   Save previous predictions and compare them with new predictions after changes -> Option D

4. Quick Check:

   Regression testing = compare old and new outputs [OK]

Hint: Keep old outputs to compare with new after changes [OK]

Common Mistakes:

• Ignoring old outputs in testing
• Assuming visualization replaces regression tests
• Deleting old results before testing