Regression testing in Testing Fundamentals - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Purpose of Regression Testing in Data Science Projects

What is the main purpose of regression testing when applied to data science models?

A To check if new code changes have broken existing model functionality

B To improve the accuracy of the model by retraining with new data

C To visualize the model predictions over time

D To optimize the hyperparameters of the model

💡 Hint

Think about what regression testing means in software and how it applies to models.

Regression testing ensures that recent changes do not break or degrade existing functionality, including model predictions.

❓ Predict Output

intermediate

Output of Regression Test on Model Predictions

Given the following Python code that compares old and new model predictions, what is the output?

Testing Fundamentals

old_preds = [0.8, 0.6, 0.9, 0.4]
new_preds = [0.8, 0.65, 0.9, 0.4]
threshold = 0.05
changes = [abs(o - n) for o, n in zip(old_preds, new_preds)]
regression_fail = any(change > threshold for change in changes)
print(regression_fail)

A True

B TypeError

C SyntaxError

D False

💡 Hint

Check if any prediction difference exceeds the threshold.

The differences are 0.0, 0.05, 0.0, 0.0. None exceed 0.05, so regression_fail is False.

❓ data_output

advanced

Detecting Regression Failures in Model Metrics

You have two sets of model evaluation metrics stored in dictionaries. Which option correctly identifies if any metric has regressed (got worse)?

Testing Fundamentals

old_metrics = {'accuracy': 0.92, 'precision': 0.85, 'recall': 0.88}
new_metrics = {'accuracy': 0.90, 'precision': 0.87, 'recall': 0.85}
regressions = {k: new_metrics[k] < old_metrics[k] for k in old_metrics}
print(regressions)

A {'accuracy': True, 'precision': True, 'recall': True}

B {'accuracy': False, 'precision': True, 'recall': False}

C {'accuracy': True, 'precision': False, 'recall': True}

D {'accuracy': False, 'precision': False, 'recall': False}

💡 Hint

Compare each metric's new value to the old value to see if it decreased.

Accuracy dropped from 0.92 to 0.90 (True), precision increased (False), recall dropped (True).

🔧 Debug

advanced

Identify the Error in Regression Test Code

What error does the following regression test code raise?

Testing Fundamentals

old_results = [0.7, 0.8, 0.9]
new_results = [0.7, 0.85]
for i in range(len(old_results)):
    if abs(old_results[i] - new_results[i]) > 0.1:
        print('Regression detected')

A IndexError

B TypeError

C No error, prints 'Regression detected'

D SyntaxError

💡 Hint

Check if the loop index matches the length of both lists.

new_results has fewer elements than old_results, so accessing new_results[2] causes IndexError.

🚀 Application

expert

Choosing the Best Regression Testing Strategy for Model Updates

You update a machine learning model frequently with new data. Which regression testing strategy best ensures that model performance does not degrade over time?

A Only retrain the model without any testing since new data should improve performance

B Run automated tests comparing new model predictions to a fixed baseline dataset and alert if differences exceed a threshold

C Manually check model outputs occasionally after major code changes

D Use random sampling of predictions without comparing to previous results

💡 Hint

Think about automation and consistent checks for regression.

Automated tests comparing predictions to a baseline help catch unintended degradations quickly and reliably.