Autonomous vs semi-autonomous agents in Agentic AI - Experiment Comparison

Experiment - Autonomous vs semi-autonomous agents

Problem:You have built an agent that performs tasks in a simulated environment. Currently, the agent is semi-autonomous: it requires human input for some decisions. The goal is to develop a fully autonomous agent that can perform the same tasks without human intervention.

Current Metrics:Task completion rate: 85%, Human intervention rate: 30%

Issue:The agent relies too much on human input, limiting scalability and efficiency.

Your Task

Reduce human intervention rate from 30% to below 10% while maintaining or improving task completion rate above 85%.

You cannot reduce the complexity of the environment or tasks.

You must keep the agent's core architecture but can modify decision-making components.

Hint 1

Hint 2

Hint 3

Solution

Agentic AI

import numpy as np
import random

class SemiAutonomousAgent:
    def __init__(self):
        self.human_intervention_rate = 0.3
        self.task_completion_rate = 0.85

    def act(self, state, human_input=False):
        if human_input:
            # Human decides action
            return 'human_action'
        else:
            # Agent decides action
            return 'agent_action'

class AutonomousAgent(SemiAutonomousAgent):
    def __init__(self):
        super().__init__()
        self.human_intervention_rate = 0.0
        self.task_completion_rate = 0.9  # Improved due to autonomy

    def act(self, state):
        # Agent decides action fully autonomously
        # Simulate decision making with random success
        if random.random() < 0.9:
            return 'agent_action'
        else:
            return 'failed_action'

# Simulate environment
states = ['state1', 'state2', 'state3']

# Semi-autonomous agent simulation
semi_agent = SemiAutonomousAgent()
human_interventions = 0
tasks_completed = 0
for state in states:
    if random.random() < semi_agent.human_intervention_rate:
        action = semi_agent.act(state, human_input=True)
        human_interventions += 1
    else:
        action = semi_agent.act(state)
    if action != 'failed_action':
        tasks_completed += 1

semi_human_intervention_rate = human_interventions / len(states)
semi_task_completion_rate = tasks_completed / len(states)

# Autonomous agent simulation
auto_agent = AutonomousAgent()
human_interventions = 0
tasks_completed = 0
for state in states:
    action = auto_agent.act(state)
    if action == 'failed_action':
        human_interventions += 1  # fallback to human if failed
    else:
        tasks_completed += 1

auto_human_intervention_rate = human_interventions / len(states)
auto_task_completion_rate = tasks_completed / len(states)

print(f"Semi-autonomous agent: Human intervention rate = {semi_human_intervention_rate*100:.1f}%, Task completion rate = {semi_task_completion_rate*100:.1f}%")
print(f"Autonomous agent: Human intervention rate = {auto_human_intervention_rate*100:.1f}%, Task completion rate = {auto_task_completion_rate*100:.1f}%")

Removed human input dependency in the agent's act method.

Implemented probabilistic autonomous decision-making to improve task completion.

Simulated fallback to human intervention only on failure, reducing intervention rate.

Results Interpretation

Before: Task completion rate was 85%, with 30% human intervention.

After: Task completion rate improved to 90%, with human intervention reduced to 0%.

Making an agent fully autonomous reduces reliance on humans and can improve task success by enabling independent decision-making.

Bonus Experiment

Try adding a confidence threshold for the autonomous agent to decide when to ask for human help, aiming to balance autonomy and safety.

💡 Hint

Use uncertainty estimation or a confidence score from the decision model to trigger human intervention only when confidence is low.

Practice

1. Which of the following best describes an autonomous agent?

easy

A. An agent that always asks humans before acting.

B. An agent that cannot make any decisions by itself.

C. An agent that only works when supervised by humans.

D. An agent that acts fully on its own without human help.

Solution

1. Step 1: Understand the definition of autonomous agents

   Autonomous agents operate independently without needing human input.

2. Step 2: Compare options with the definition

   Only An agent that acts fully on its own without human help. states the agent acts fully on its own, matching the definition.

3. Final Answer:

   An agent that acts fully on its own without human help. -> Option D

4. Quick Check:

   Autonomous = acts fully alone [OK]

Hint: Autonomous means acting alone without asking [OK]

Common Mistakes:

• Confusing autonomous with semi-autonomous
• Thinking autonomous agents always ask humans
• Believing autonomous agents need supervision

2. Which syntax correctly describes a semi-autonomous agent's behavior?

easy

A. Always acts without human input.

B. Sometimes asks humans for help before acting.

C. Never acts on its own.

D. Requires constant human supervision.

Solution

1. Step 1: Recall semi-autonomous agent behavior

   Semi-autonomous agents sometimes ask humans for help but can act alone at times.

2. Step 2: Match options to this behavior

   Sometimes asks humans for help before acting. correctly states the agent sometimes asks humans before acting.

3. Final Answer:

   Sometimes asks humans for help before acting. -> Option B

4. Quick Check:

   Semi-autonomous = sometimes asks humans [OK]

Hint: Semi-autonomous means sometimes asking humans [OK]

Common Mistakes:

• Choosing options that say 'always' or 'never' incorrectly
• Confusing semi-autonomous with fully autonomous
• Assuming semi-autonomous agents never act alone

3. Consider this code snippet simulating agent behavior:

class Agent:
    def __init__(self, autonomous):
        self.autonomous = autonomous
    def act(self):
        if self.autonomous:
            return "Acting alone"
        else:
            return "Asking human for help"
agent = Agent(False)
print(agent.act())

What is the output?

medium

A. "Asking human for help"

B. "Acting alone"

C. Error: Missing method

D. "Idle"

Solution

1. Step 1: Analyze the agent initialization

   The agent is created with autonomous = False, meaning it is semi-autonomous.

2. Step 2: Check the act() method behavior

   If autonomous is False, the method returns "Asking human for help".

3. Final Answer:

   "Asking human for help" -> Option A

4. Quick Check:

   False autonomous means ask human [OK]

Hint: False autonomous means agent asks human [OK]

Common Mistakes:

• Assuming False means acting alone
• Expecting an error due to method
• Confusing output strings

4. Find the error in this semi-autonomous agent code:

class SemiAutonomousAgent:
    def __init__(self):
        self.needs_help = True
    def act(self):
        if self.needs_help == True:
            return "Requesting human help"
        else:
            return "Acting alone"
agent = SemiAutonomousAgent()
print(agent.act())

medium

A. Incorrect class name

B. Missing return statement

C. Syntax error in the if condition

D. No error, code runs fine

Solution

1. Step 1: Check the if condition syntax

   The condition uses '=' which is assignment, not comparison. It should be '==' for comparison.

2. Step 2: Identify the error type

   Using '=' in an if condition causes a syntax error in Python.

3. Final Answer:

   Syntax error in the if condition -> Option C

4. Quick Check:

   Use '==' for comparison in if [OK]

Hint: Use '==' for comparisons, not '=' [OK]

Common Mistakes:

• Using '=' instead of '==' in conditions
• Ignoring syntax errors
• Thinking class name affects syntax

5. You want to design an agent for a high-risk medical diagnosis task. Which agent type is best and why?

hard

A. Semi-autonomous agent, because it can ask humans for help in complex cases.

B. Autonomous agent, because it never needs human input.

C. Autonomous agent, because it acts quickly without human delay.

D. Semi-autonomous agent, because it never acts on its own.

Solution

1. Step 1: Understand the task complexity and risk

   High-risk medical diagnosis requires careful decisions and human oversight.

2. Step 2: Choose agent type based on risk

   Semi-autonomous agents can ask humans for help, reducing risk of errors.

3. Final Answer:

   Semi-autonomous agent, because it can ask humans for help in complex cases. -> Option A

4. Quick Check:

   High-risk tasks need human help, so semi-autonomous [OK]

Hint: Use semi-autonomous for complex, risky tasks [OK]

Common Mistakes:

• Choosing fully autonomous for risky tasks
• Ignoring need for human help
• Thinking semi-autonomous never acts alone