Regression testing for agent changes in Agentic AI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why is regression testing important after updating an AI agent?

Imagine you have an AI agent that helps users book flights. You update its code to improve speed. Why should you run regression tests after this update?

A To add new features to the agent

B To check if the update accidentally broke existing features

C To improve the agent's speed further

D To delete old data from the system

💡 Hint

Think about what might happen to old features when you change something new.

Regression testing ensures that new changes do not break existing functionalities. It helps keep the agent reliable.

❓ Predict Output

intermediate

Output of regression test result summary

Given this Python code simulating regression test results for an AI agent update, what will be printed?

Agentic AI

test_results = {'login': True, 'search': True, 'booking': False, 'payment': True}
failed_tests = [k for k, v in test_results.items() if not v]
print(f"Failed tests: {failed_tests}")

A Failed tests: ['login', 'search']

B Failed tests: ['payment']

C Failed tests: []

D Failed tests: ['booking']

💡 Hint

Look for tests with value False in the dictionary.

The code collects keys where the test result is False. Only 'booking' test failed.

❓ Model Choice

advanced

Choosing a model for regression testing of agent behavior

You want to predict if an AI agent update will cause failures in certain tasks based on past update data. Which model is best suited for this binary classification problem?

A Linear Regression

B K-Means Clustering

C Logistic Regression

D Principal Component Analysis (PCA)

💡 Hint

Think about models that predict categories like pass/fail.

Logistic Regression is used for binary classification problems like predicting pass/fail outcomes.

❓ Metrics

advanced

Interpreting regression test classification metrics

An AI agent regression test classifier has these results: 90 true positives, 10 false positives, 5 false negatives, and 95 true negatives. What is the precision of the classifier?

A 90 / (90 + 10) = 0.9

B 90 / (90 + 5) = 0.947

C 95 / (95 + 10) = 0.905

D 90 / (90 + 95) = 0.486

💡 Hint

Precision = True Positives / (True Positives + False Positives)

Precision measures how many predicted positives are actually correct.

🔧 Debug

expert

Debugging a failing regression test script for agent changes

Consider this Python code snippet that runs regression tests on an AI agent's functions. It should print 'All tests passed' if all tests return True, else print failed test names. What error or output will this code produce?

Agentic AI

def test_login():
    return True

def test_search():
    return False

tests = {'login': test_login, 'search': test_search}

failed = []
for name, func in tests.items():
    if func() == False:
        failed.append(name)

if failed:
    print('Failed tests:', failed)
else:
    print('All tests passed')

A Failed tests: ['search']

B All tests passed

C TypeError because func() is not callable

D SyntaxError due to missing colon

💡 Hint

Check which test returns False and how the code collects failures.

The 'search' test returns False, so it is added to failed list and printed.