Computer use agents in Agentic AI - ML Experiment: Train & Evaluate

Experiment - Computer use agents

Problem:We want to build a computer agent that can learn to use a simple computer interface to complete tasks like opening files or clicking buttons.

Current Metrics:Training success rate: 98%, Validation success rate: 65%

Issue:The agent is overfitting: it performs very well on training tasks but poorly on new, unseen tasks.

Your Task

Reduce overfitting so that the validation success rate improves to at least 85%, while keeping training success rate below 95%.

You can only change the agent's training method and model architecture.

You cannot change the task environment or the data itself.

Hint 1

Hint 2

Hint 3

Solution

Agentic AI

import tensorflow as tf
from tensorflow.keras import layers, models

# Define the agent model with dropout to reduce overfitting
def create_agent_model():
    model = models.Sequential([
        layers.Dense(128, activation='relu', input_shape=(input_dim,)),
        layers.Dropout(0.5),
        layers.Dense(64, activation='relu'),
        layers.Dropout(0.3),
        layers.Dense(output_dim, activation='softmax')
    ])
    model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
    return model

# Assume X_train, y_train, X_val, y_val are prepared datasets

input_dim = 100  # example input size
output_dim = 10  # example output size

agent_model = create_agent_model()

# Use early stopping to stop training when validation loss stops improving
early_stop = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=5, restore_best_weights=True)

history = agent_model.fit(
    X_train, y_train,
    epochs=50,
    batch_size=32,
    validation_data=(X_val, y_val),
    callbacks=[early_stop]
)

# Evaluate final performance
train_loss, train_acc = agent_model.evaluate(X_train, y_train, verbose=0)
val_loss, val_acc = agent_model.evaluate(X_val, y_val, verbose=0)

print(f'Training accuracy: {train_acc*100:.2f}%')
print(f'Validation accuracy: {val_acc*100:.2f}%')

Added dropout layers after dense layers to reduce overfitting by randomly turning off neurons during training.

Implemented early stopping to halt training when validation loss stops improving, preventing memorization.

Kept model size moderate with two dense layers to avoid excessive complexity.

Results Interpretation

Before: Training success rate was 98%, validation success rate was 65%. The agent memorized training tasks but failed on new ones.

After: Training success rate reduced to 92%, validation success rate improved to 87%. The agent generalized better to new tasks.

Adding dropout and early stopping helps reduce overfitting by preventing the agent from memorizing training data, leading to better performance on new tasks.

Bonus Experiment

Try using data augmentation by slightly modifying the input states during training to improve generalization further.

💡 Hint

Introduce small random changes to the input features to simulate different computer states, helping the agent learn more robust behaviors.

Practice

1. What is the main role of a computer use agent?

easy

A. To display graphics on the screen

B. To perform tasks automatically by sensing and acting

C. To store large amounts of data

D. To manually control the computer hardware

Solution

1. Step 1: Understand what an agent does

   An agent senses its environment and takes actions to complete tasks automatically.

2. Step 2: Compare options with this definition

   Only To perform tasks automatically by sensing and acting describes automatic task performance by sensing and acting.

3. Final Answer:

   To perform tasks automatically by sensing and acting -> Option B

4. Quick Check:

   Agent role = automatic task performance [OK]

Hint: Agents act automatically by sensing environment [OK]

Common Mistakes:

• Confusing agents with hardware controllers
• Thinking agents only store data
• Assuming agents only display information

2. Which of the following is the correct way to describe an agent's action cycle?

easy

A. Sense environment -> Take action -> Update environment

B. Take action -> Sense environment -> Sleep

C. Sense environment -> Sleep -> Take action

D. Update environment -> Take action -> Sense environment

Solution

1. Step 1: Recall the agent cycle steps

   An agent first senses its environment, then takes an action based on that sensing.

2. Step 2: Match the correct sequence

   Sense environment -> Take action -> Update environment correctly shows sensing first, then acting, then environment update.

3. Final Answer:

   Sense environment -> Take action -> Update environment -> Option A

4. Quick Check:

   Agent cycle = sense then act [OK]

Hint: Agents sense first, then act, then update [OK]

Common Mistakes:

• Mixing order of sensing and acting
• Including sleep incorrectly in cycle
• Ignoring environment update step

3. Consider this simple agent code snippet:

class Agent:
    def __init__(self):
        self.state = 0
    def sense(self, input):
        self.state += input
    def act(self):
        return self.state * 2

agent = Agent()
agent.sense(3)
agent.sense(4)
print(agent.act())

What is the output of this code?

medium

A. 14

B. 7

C. 12

D. 0

Solution

1. Step 1: Calculate state after sensing inputs

   Initial state is 0. After agent.sense(3), state = 3. After agent.sense(4), state = 7.

2. Step 2: Calculate action output

   agent.act() returns state * 2 = 7 * 2 = 14.

3. Final Answer:

   14 -> Option A

4. Quick Check:

   State sum 7 x 2 = 14 [OK]

Hint: Add inputs then multiply by 2 for output [OK]

Common Mistakes:

• Multiplying inputs separately instead of sum
• Using only last input instead of sum
• Confusing state update logic

4. This agent code has a bug:

class Agent:
    def __init__(self):
        self.state = 0
    def sense(self, input):
        self.state = input
    def act(self):
        return self.state * 2

agent = Agent()
agent.sense(3)
agent.sense(4)
print(agent.act())

What is the bug and how to fix it?

medium

A. Bug: sense method missing; Fix: add sense method

B. Bug: act returns wrong value; Fix: return state + 2

C. Bug: state overwritten each sense; Fix: use += to accumulate

D. Bug: state not initialized; Fix: initialize state in act

Solution

1. Step 1: Identify the problem in sense method

   The sense method sets state = input, so previous state is lost on each call.

2. Step 2: Fix by accumulating inputs

   Change state = input to state += input to keep adding inputs.

3. Final Answer:

   Bug: state overwritten each sense; Fix: use += to accumulate -> Option C

4. Quick Check:

   Accumulate inputs with += fixes bug [OK]

Hint: Use += to add inputs, not = to overwrite [OK]

Common Mistakes:

• Thinking act method is wrong
• Adding sense method again unnecessarily
• Initializing state in wrong place

5. You want to design a smart agent that automatically adjusts room temperature based on sensor data. Which approach best fits this task?

hard

A. Use a simple reflex agent that acts only on current sensor reading

B. Use a fixed schedule agent ignoring sensor data

C. Use a random agent that changes temperature randomly

D. Use a model-based agent that keeps track of past temperatures

Solution

1. Step 1: Understand task needs

   Adjusting temperature smartly requires remembering past data to avoid sudden changes.

2. Step 2: Choose agent type

   A model-based agent keeps track of past states, making it suitable for this task.

3. Final Answer:

   Use a model-based agent that keeps track of past temperatures -> Option D

4. Quick Check:

   Smart adjustment needs model-based agent [OK]

Hint: Smart agents remember past data for better decisions [OK]

Common Mistakes:

• Choosing simple reflex agent ignoring history
• Using random or fixed schedule agents
• Not considering past sensor data