Why guardrails prevent agent disasters in Agentic AI - Experiment to Prove It

Experiment - Why guardrails prevent agent disasters

Problem:You have an AI agent designed to perform tasks autonomously. However, without safety guardrails, the agent sometimes takes harmful or unintended actions.

Current Metrics:Agent success rate: 85%, but 15% of actions cause unintended harmful side effects.

Issue:The agent performs well on tasks but occasionally causes disasters due to lack of constraints or safety checks.

Your Task

Add guardrails to the agent to reduce harmful side effects from 15% to below 5%, while maintaining at least 85% task success rate.

You cannot reduce the agent's task capabilities.

You must keep the agent's response time within 10% of the original.

Hint 1

Hint 2

Hint 3

Solution

Agentic AI

import random

class Agent:
    def __init__(self):
        self.task_success_rate = 0.85
        self.harmful_action_rate = 0.15

    def act(self):
        # Simulate action with chance of harm
        if random.random() < self.harmful_action_rate:
            return 'harmful_action'
        else:
            return 'safe_action'

class GuardrailAgent(Agent):
    def __init__(self):
        super().__init__()
        self.harmful_action_rate = 0.15

    def safety_check(self, action):
        # Simple guardrail: block harmful actions
        if action == 'harmful_action':
            return 'blocked_action'
        return action

    def act(self):
        action = super().act()
        safe_action = self.safety_check(action)
        return safe_action

# Evaluate agent before guardrails
agent = Agent()
trials = 1000
harmful_count = 0
success_count = 0
for _ in range(trials):
    action = agent.act()
    if action == 'harmful_action':
        harmful_count += 1
    else:
        success_count += 1

# Evaluate agent after guardrails
guardrail_agent = GuardrailAgent()
harmful_count_gr = 0
success_count_gr = 0
blocked_count = 0
for _ in range(trials):
    action = guardrail_agent.act()
    if action == 'harmful_action':
        harmful_count_gr += 1
    elif action == 'blocked_action':
        blocked_count += 1
    else:
        success_count_gr += 1

print(f"Before guardrails: Success rate = {success_count/trials*100:.1f}%, Harmful actions = {harmful_count/trials*100:.1f}%")
print(f"After guardrails: Success rate = {success_count_gr/trials*100:.1f}%, Harmful actions = {harmful_count_gr/trials*100:.1f}%, Blocked actions = {blocked_count/trials*100:.1f}%")

Added a safety_check method to block harmful actions before execution.

Modified the act method to apply safety_check and prevent harmful actions.

Kept task success rate high by only blocking harmful actions, not safe ones.

Results Interpretation

Before guardrails: Success rate 85%, Harmful actions 15%

After guardrails: Success rate 85%, Harmful actions 0%, Blocked actions 15%

Adding guardrails prevents harmful actions without reducing the agent's ability to complete tasks. This shows how safety checks help avoid disasters while keeping performance.

Bonus Experiment

Try adding a penalty in the agent's learning process for harmful actions instead of blocking them outright.

💡 Hint

Modify the reward function to reduce rewards when harmful actions occur, encouraging the agent to learn safer behavior.

Practice

1. Why are guardrails important for AI agents when they interact with people?

easy

A. They make the AI run faster.

B. They help the AI learn without any rules.

C. They allow the AI to ignore user input.

D. They prevent the AI from making harmful or unsafe decisions.

Solution

1. Step 1: Understand the role of guardrails

   Guardrails are safety limits set to stop AI from doing harmful actions.

2. Step 2: Connect guardrails to interaction with people

   When AI talks to people, guardrails keep it from unsafe or harmful choices.

3. Final Answer:

   They prevent the AI from making harmful or unsafe decisions. -> Option D

4. Quick Check:

   Guardrails = prevent harm [OK]

Hint: Guardrails stop bad AI actions with people [OK]

Common Mistakes:

• Thinking guardrails speed up AI
• Believing guardrails ignore user input
• Assuming guardrails remove all rules

2. Which of the following is the correct way to add a guardrail that stops an AI agent from deleting files?

easy

A. delete_file = true

B. allow action == 'delete_file'

C. if action == 'delete_file': block()

D. action = 'delete_file'

Solution

1. Step 1: Identify guardrail syntax to block actions

   The guardrail should check if the action is 'delete_file' and then block it.

2. Step 2: Compare options for correct blocking

   if action == 'delete_file': block() uses a condition and blocks the action, which is correct for a guardrail.

3. Final Answer:

   if action == 'delete_file': block() -> Option C

4. Quick Check:

   Guardrail blocks delete_file = if action == 'delete_file': block() [OK]

Hint: Guardrails use conditions to block bad actions [OK]

Common Mistakes:

• Allowing the action instead of blocking
• Assigning variables instead of checking conditions
• Confusing action names with commands

3. Given this code snippet for an AI agent guardrail:

actions = ['read_data', 'delete_file', 'send_email']
allowed_actions = []
for a in actions:
    if a != 'delete_file':
        allowed_actions.append(a)
print(allowed_actions)

What will be the output?

medium

A. ['read_data', 'delete_file', 'send_email']

B. ['read_data', 'send_email']

C. ['delete_file']

D. []

Solution

1. Step 1: Understand the loop and condition

   The loop goes through each action and adds it to allowed_actions only if it is not 'delete_file'.

2. Step 2: Trace the loop with given actions

   'read_data' is added, 'delete_file' is skipped, 'send_email' is added.

3. Final Answer:

   ['read_data', 'send_email'] -> Option B

4. Quick Check:

   Filtered out 'delete_file' = ['read_data', 'send_email'] [OK]

Hint: Check which actions pass the condition [OK]

Common Mistakes:

• Including 'delete_file' by mistake
• Empty list if loop misunderstood
• Confusing append with replace

4. This AI agent code is meant to block unsafe commands but has a bug:

def guardrail(action):
    if action = 'shutdown':
        return 'Blocked'
    else:
        return 'Allowed'

What is the error and how to fix it?

medium

A. Use '==' instead of '=' in the if condition.

B. Change 'return' to 'print' inside the function.

C. Remove the else block entirely.

D. Add a colon ':' after the function name.

Solution

1. Step 1: Identify the syntax error in the if statement

   The code uses '=' which is assignment, but it should compare with '==' in conditions.

2. Step 2: Correct the if condition to use '=='

   Replace '=' with '==' to properly check if action equals 'shutdown'.

3. Final Answer:

   Use '==' instead of '=' in the if condition. -> Option A

4. Quick Check:

   Comparison needs '==' [OK]

Hint: Use '==' for comparison, '=' is assignment [OK]

Common Mistakes:

• Confusing assignment '=' with comparison '=='
• Changing return to print unnecessarily
• Removing else block without reason

5. An AI agent is designed to handle user requests but must never share private data. Which guardrail strategy best prevents accidental data leaks?

hard

A. Filter all outputs to remove sensitive keywords before sending.

B. Allow all outputs but log them for review later.

C. Ignore user requests that mention private data without warning.

D. Let the AI decide case-by-case if data is private.

Solution

1. Step 1: Understand the goal to prevent data leaks

   The guardrail must stop private data from being shared in outputs.

2. Step 2: Evaluate options for effective prevention

   Filtering outputs to remove sensitive keywords directly blocks leaks, unlike logging or ignoring.

3. Final Answer:

   Filter all outputs to remove sensitive keywords before sending. -> Option A

4. Quick Check:

   Filtering outputs = safest guardrail [OK]

Hint: Filter outputs to block private data leaks [OK]

Common Mistakes:

• Relying only on logs without blocking
• Ignoring requests silently
• Trusting AI to decide privacy alone