Defining success criteria for agents in Agentic AI - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to define a success criterion function that returns True if the agent's score is above 80.

Agentic AI

def success_criterion(score):
    return score [1] 80

Drag options to blanks, or click blank then click option'

A<

B>

C==

D<=

Attempts:

3 left

💡 Hint

Common Mistakes

Using '<' instead of '>' will mark low scores as success.

Using '==' is too strict and only accepts exactly 80.

The success criterion checks if the score is greater than 80, so the correct operator is '>'.

2fill in blank

medium

Complete the code to define a function that returns True if the agent's task completion time is less than the allowed time.

Agentic AI

def success_criterion(completion_time, allowed_time):
    return completion_time [1] allowed_time

Drag options to blanks, or click blank then click option'

A<

B>

C>=

D==

Attempts:

3 left

💡 Hint

Common Mistakes

Using '>' would mean finishing after the allowed time is success.

Using '==' is too strict and only accepts exact matches.

The agent succeeds if it finishes before the allowed time, so completion_time must be less than allowed_time.

3fill in blank

hard

Fix the error in the success check that should return True if the agent's accuracy is at least 90%.

Agentic AI

def success_criterion(accuracy):
    return accuracy [1] 0.9

Drag options to blanks, or click blank then click option'

A>=

B<

C<=

D==

Attempts:

3 left

💡 Hint

Common Mistakes

Using '<' or '<=' would mark low accuracy as success.

Using '==' only accepts exactly 0.9 accuracy.

The success criterion requires accuracy to be at least 90%, so '>=' is the correct operator.

4fill in blank

hard

Fill both blanks to create a success criterion that returns True if the agent's reward is greater than 50 and the error is less than 0.1.

Agentic AI

def success_criterion(reward, error):
    return reward [1] 50 and error [2] 0.1

Drag options to blanks, or click blank then click option'

A>

B<

C>=

D<=

Attempts:

3 left

💡 Hint

Common Mistakes

Using '>=' or '<=' changes the strictness of the condition.

Mixing up the operators reverses the logic.

The agent succeeds if reward is greater than 50 and error is less than 0.1, so the operators are '>' and '<'.

5fill in blank

hard

Fill all three blanks to define a success criterion that returns True if the agent's score is at least 75, time is less than 120, and errors are at most 5.

Agentic AI

def success_criterion(score, time, errors):
    return score [1] 75 and time [2] 120 and errors [3] 5

Drag options to blanks, or click blank then click option'

A>=

B<

C<=

D>

Attempts:

3 left

💡 Hint

Common Mistakes

Using '>' instead of '>=' excludes exact 75 score.

Using '>' or '>=' for time or errors reverses the logic.

The agent succeeds if score >= 75, time < 120, and errors <= 5, so the operators are '>=', '<', and '<=' respectively.

Practice

1. Why is it important to define success criteria for an AI agent?

easy

A. It reduces the size of the agent's code.

B. It helps the agent understand what goal to achieve.

C. It makes the agent run faster.

D. It allows the agent to ignore errors.

Solution

1. Step 1: Understand the role of success criteria

   Success criteria tell the agent what outcome is desired or considered good.

2. Step 2: Connect success criteria to agent behavior

   Without clear goals, the agent cannot know what to aim for or when it has succeeded.

3. Final Answer:

   It helps the agent understand what goal to achieve. -> Option B

4. Quick Check:

   Success criteria = clear goals [OK]

Hint: Success criteria define the agent's goal clearly [OK]

Common Mistakes:

• Thinking success criteria speed up the agent
• Confusing success criteria with code size
• Believing success criteria ignore errors

2. Which of the following is the correct way to express a success criterion for an agent in code?

easy

A. success == accuracy > 0.9

B. success = accuracy = 0.9

C. success = accuracy > 0.9

D. success => accuracy > 0.9

Solution

1. Step 1: Identify correct comparison syntax

   In Python, to assign a boolean result, use a single = with a comparison expression on the right.

2. Step 2: Check each option's syntax

   success = accuracy > 0.9 uses correct assignment and comparison. success = accuracy = 0.9 uses = instead of == for comparison. success == accuracy > 0.9 uses == incorrectly for assignment. success => accuracy > 0.9 uses => which is invalid in Python.

3. Final Answer:

   success = accuracy > 0.9 -> Option C

4. Quick Check:

   Assignment with comparison uses = and > [OK]

Hint: Use '=' for assignment, '>' for comparison [OK]

Common Mistakes:

• Using '==' instead of '=' for assignment
• Using '=' instead of '==' for comparison
• Using invalid operators like '=>'

3. Given the code below, what will be the value of success?

accuracy = 0.85
threshold = 0.8
success = accuracy >= threshold

medium

A. True

B. Error

C. 0.85

D. False

Solution

1. Step 1: Compare accuracy and threshold values

   Accuracy is 0.85, threshold is 0.8, so 0.85 >= 0.8 is True.

2. Step 2: Assign comparison result to success

   The boolean True is assigned to success.

3. Final Answer:

   True -> Option A

4. Quick Check:

   0.85 >= 0.8 = True [OK]

Hint: Check if accuracy meets or exceeds threshold [OK]

Common Mistakes:

• Confusing value 0.85 with boolean True
• Thinking comparison returns a number
• Expecting an error from valid comparison

4. The following code is intended to check if an agent's success metric is above 90%, but it has a bug. What is the bug?

success_metric = 0.92
if success_metric = 0.9:
    print('Agent succeeded')

medium

A. Missing colon ':' after if statement

B. Print statement syntax error

C. Incorrect variable name 'success_metric'

D. Using '=' instead of '==' in the if condition

Solution

1. Step 1: Identify the if statement syntax

   In Python, '=' is for assignment, '==' is for comparison in conditions.

2. Step 2: Locate the bug in the if condition

   The code uses '=' instead of '==' which causes a syntax error.

3. Final Answer:

   Using '=' instead of '==' in the if condition -> Option D

4. Quick Check:

   Use '==' for comparison in if [OK]

Hint: Use '==' to compare values in if statements [OK]

Common Mistakes:

• Confusing '=' with '==' in conditions
• Ignoring syntax errors from wrong operators
• Assuming missing colon is the error

5. You want to define success criteria for an agent that completes tasks with at least 95% accuracy and finishes within 10 seconds. Which of the following is the best way to define this success criteria in code?

hard

A. success = (accuracy >= 0.95) and (time_taken <= 10)

B. success = accuracy > 0.95 or time_taken < 10

C. success = accuracy == 0.95 and time_taken == 10

D. success = accuracy >= 0.95 and time_taken > 10

Solution

1. Step 1: Understand the criteria requirements

   The agent must have accuracy at least 95% and finish within 10 seconds.

2. Step 2: Translate criteria into logical conditions

   Use '>=' for accuracy and '<=' for time, combined with 'and' to require both.

3. Step 3: Evaluate each option

   success = (accuracy >= 0.95) and (time_taken <= 10) correctly uses 'and' and proper comparisons. success = accuracy > 0.95 or time_taken < 10 uses 'or' which allows passing if only one condition is met. success = accuracy == 0.95 and time_taken == 10 uses '==' which is too strict. success = accuracy >= 0.95 and time_taken > 10 allows time_taken > 10 which breaks the time limit.

4. Final Answer:

   success = (accuracy >= 0.95) and (time_taken <= 10) -> Option A

5. Quick Check:

   Both accuracy and time must meet thresholds [OK]

Hint: Use 'and' to combine all success conditions [OK]

Common Mistakes:

• Using 'or' instead of 'and' to combine conditions
• Using '==' instead of '>=' or '<='
• Allowing time greater than limit