Regression testing for agent changes in Agentic AI - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to run a regression test on the agent's output.

Agentic AI

assert agent_output == [1]

Drag options to blanks, or click blank then click option'

Anew_output

Bagent_output

Ctest_result

Dexpected_output

Attempts:

3 left

💡 Hint

Common Mistakes

Comparing the output to itself instead of the expected result.

Using an undefined variable for comparison.

We compare the agent's output to the expected output to verify correctness.

2fill in blank

medium

Complete the code to log the regression test result.

Agentic AI

logger.[1]('Regression test passed')

Drag options to blanks, or click blank then click option'

Ainfo

Bdebug

Cwarn

Derror

Attempts:

3 left

💡 Hint

Common Mistakes

Using error or warn logging for passing tests.

Using debug which might be too low level.

Logging info level is appropriate for successful test messages.

3fill in blank

hard

Fix the error in the regression test function call.

Agentic AI

def test_agent():
    result = agent.run(input_data)
    assert result [1] expected_output

Drag options to blanks, or click blank then click option'

A!=

B>=

C==

D<=

Attempts:

3 left

💡 Hint

Common Mistakes

Using '!=' which checks for inequality.

Using comparison operators like '>=' or '<=' which are not suitable.

The test asserts that the result equals the expected output.

4fill in blank

hard

Fill both blanks to create a dictionary of test results with agent names and pass status.

Agentic AI

test_results = {agent_name: [1] for agent_name in agents if agents[agent_name].active [2] True}

Drag options to blanks, or click blank then click option'

Aagents[agent_name].run(input_data) == expected_output

Bagent.run(input_data)

C==

D!=

Attempts:

3 left

💡 Hint

Common Mistakes

Using '!=' instead of '==' in the condition.

Using the agent.run(input_data) without comparison in the dictionary.

We check if each agent's run output equals the expected output and filter agents that are active (== True).

5fill in blank

hard

Fill all three blanks to filter agents, run tests, and store pass status in a dictionary.

Agentic AI

results = {agent: (output := agent.run(data)) [1] expected for agent, expected in [2] if agent.active == [3]

Drag options to blanks, or click blank then click option'

A==

Bitems()

CTrue

D!=

Attempts:

3 left

💡 Hint

Common Mistakes

Using '!=' instead of '==' for comparison.

Using keys() instead of items() for iteration.

Checking if agent.active is False instead of True.

We compare output to expected with '==', iterate over items(), and check if agent is active (True).

Practice

1. What is the main purpose of regression testing for agent changes?

easy

A. To check if new changes break old agent behavior

B. To improve the agent's speed

C. To add new features to the agent

D. To change the agent's user interface

Solution

1. Step 1: Understand regression testing goal

   Regression testing is done to ensure that recent changes do not break existing functionality.

2. Step 2: Match purpose with options

   To check if new changes break old agent behavior clearly states checking if new changes break old behavior, which matches the goal.

3. Final Answer:

   To check if new changes break old agent behavior -> Option A

4. Quick Check:

   Regression testing = check old behavior intact [OK]

Hint: Regression testing checks old features still work after changes [OK]

Common Mistakes:

• Thinking regression testing adds new features
• Confusing regression testing with performance testing
• Assuming regression testing changes UI

2. Which of the following is the correct way to define a test case for regression testing an agent in Python?

easy

A. def test_agent(): assert agent.run(input) == expected_output

B. test agent run input equals expected output

C. def test_agent: return agent.run(input) == expected_output

D. function test_agent() { return agent.run(input) == expected_output; }

Solution

1. Step 1: Identify correct Python function syntax

   Python functions start with 'def', have parentheses, and a colon.

2. Step 2: Check assertion usage

   def test_agent(): assert agent.run(input) == expected_output uses 'assert' correctly to compare output, matching Python test style.

3. Final Answer:

   def test_agent(): assert agent.run(input) == expected_output -> Option A

4. Quick Check:

   Python test function with assert = def test_agent(): assert agent.run(input) == expected_output [OK]

Hint: Python test functions start with def and use assert [OK]

Common Mistakes:

• Missing parentheses or colon in function definition
• Using non-Python syntax
• Not using assert for test checks

3. Given the code below, what will be the output of the regression test?

class Agent:
    def run(self, x):
        return x * 2

def test_agent():
    agent = Agent()
    result = agent.run(3)
    assert result == 6
    print('Test passed')

test_agent()

medium

A. SyntaxError

B. Test passed

C. AssertionError

D. No output

Solution

1. Step 1: Understand agent run method

   The method multiplies input by 2, so run(3) returns 6.

2. Step 2: Check assertion and print

   The assertion checks if result == 6, which is true, so no error occurs and 'Test passed' prints.

3. Final Answer:

   Test passed -> Option B

4. Quick Check:

   3 * 2 = 6, assertion true, prints message [OK]

Hint: Check method output matches assertion to predict test result [OK]

Common Mistakes:

• Assuming assertion fails without checking output
• Confusing syntax errors with logic errors
• Ignoring print statement after assertion

4. Identify the error in the following regression test code and select the fix:

def test_agent():
    agent = Agent()
    result = agent.run(5)
    if result = 10:
        print('Test passed')
    else:
        print('Test failed')

medium

A. Replace print with return statements

B. Add parentheses around the if condition

C. Change '=' to '==' in the if condition

D. Remove else block

Solution

1. Step 1: Identify syntax error in if condition

   The single '=' is an assignment, not a comparison, causing a syntax error.

2. Step 2: Correct the comparison operator

   Replace '=' with '==' to compare values properly in the if statement.

3. Final Answer:

   Change '=' to '==' in the if condition -> Option C

4. Quick Check:

   Use '==' for comparison in if statements [OK]

Hint: Use '==' to compare, '=' assigns values [OK]

Common Mistakes:

• Using '=' instead of '==' in conditions
• Adding unnecessary parentheses in Python if
• Thinking print must be replaced with return

5. You updated your agent's decision logic. How should you design regression tests to ensure old behaviors remain correct while testing new features?

hard

A. Test randomly without expected outputs to save time

B. Only test new features since old ones worked before

C. Remove old tests to avoid conflicts with new logic

D. Create test cases for both old expected outputs and new expected outputs

Solution

1. Step 1: Understand regression test purpose

   Regression tests verify that old behaviors still work after changes.

2. Step 2: Design tests covering old and new behaviors

   Include test cases for old expected outputs and new expected outputs to check both.

3. Final Answer:

   Create test cases for both old expected outputs and new expected outputs -> Option D

4. Quick Check:

   Test old and new outputs to ensure full correctness [OK]

Hint: Test old and new cases to catch breaks early [OK]

Common Mistakes:

• Ignoring old tests after updates
• Deleting old tests to simplify
• Skipping expected outputs in tests