Self-improving agents in Agentic AI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is a key characteristic of a self-improving agent?

Which of the following best describes a self-improving agent in AI?

A An agent that randomly changes its behavior without evaluation.

B An agent that requires manual updates from developers to improve.

C An agent that can modify its own code or parameters to improve performance over time.

D An agent that only follows fixed rules without any learning capability.

💡 Hint

Think about what makes an agent improve itself without external help.

Self-improving agents can autonomously adjust their own code or parameters to enhance their performance, unlike fixed or manually updated agents.

❓ Model Choice

intermediate

Choosing a model architecture for a self-improving agent

You want to build a self-improving agent that learns from its past decisions and adapts its strategy. Which model architecture is most suitable?

A A reinforcement learning model with policy gradient methods.

B A simple linear regression model without feedback.

C A clustering algorithm that groups data points.

D A static decision tree with fixed rules.

💡 Hint

Consider models that learn from interaction and improve policies over time.

Reinforcement learning models with policy gradients allow agents to learn from feedback and improve their decision-making policies, which is essential for self-improvement.

❓ Metrics

advanced

Evaluating improvement in a self-improving agent

Which metric is most appropriate to measure the improvement of a self-improving agent over multiple training episodes?

A Average cumulative reward per episode.

B Mean squared error on a fixed test set.

C Number of parameters in the model.

D Training time per epoch.

💡 Hint

Think about a metric that reflects how well the agent performs its task over time.

Average cumulative reward per episode directly measures how well the agent is performing and improving in its environment over time.

🔧 Debug

advanced

Debugging a self-improving agent stuck in local optimum

An agent using reinforcement learning is stuck with poor performance and does not improve despite training. What is the most likely cause?

A The agent's model architecture is too complex.

B The agent's learning rate is too high, causing unstable updates.

C The reward function is too sparse, providing no feedback.

D The agent's exploration rate is too low, causing it to exploit suboptimal actions.

💡 Hint

Consider why the agent might not try new actions to find better solutions.

If the exploration rate is too low, the agent mostly exploits known actions and fails to discover better strategies, causing it to get stuck in a local optimum.

❓ Predict Output

expert

Output of a self-modifying agent code snippet

Consider this Python code simulating a simple self-improving agent that updates its parameter to maximize reward. What is the output after running it?

Agentic AI

class SelfImprovingAgent:
    def __init__(self):
        self.param = 0
    def reward(self):
        return -(self.param - 5) ** 2 + 10
    def improve(self):
        best_param = self.param
        best_reward = self.reward()
        for delta in [-1, 1]:
            candidate = self.param + delta
            candidate_reward = -(candidate - 5) ** 2 + 10
            if candidate_reward > best_reward:
                best_param = candidate
                best_reward = candidate_reward
        self.param = best_param
agent = SelfImprovingAgent()
for _ in range(3):
    agent.improve()
print(agent.param)

A 4

B 3

C 5

D 2

💡 Hint

Trace the parameter updates step by step to see where it converges after 3 improvements.

The agent starts at param=0 (reward=-15). Each improve step moves to the neighbor with higher reward: 0→1 (-6 > -15), 1→2 (1 > -6), 2→3 (6 > 1). After 3 steps, param=3.

Practice

1. What is the main idea behind a self-improving agent in AI?

easy

A. It learns from its own actions to get better over time.

B. It only follows fixed rules without changing.

C. It requires constant manual updates to improve.

D. It ignores feedback from the environment.

Solution

1. Step 1: Understand the agent's learning process

   A self-improving agent learns by trying actions and observing results to improve itself.

2. Step 2: Compare options to the definition

   Only It learns from its own actions to get better over time. describes learning from its own actions to improve over time.

3. Final Answer:

   It learns from its own actions to get better over time. -> Option A

4. Quick Check:

   Self-improving means learning from actions = B [OK]

Hint: Self-improving means learning and updating itself [OK]

Common Mistakes:

• Thinking it never changes (fixed rules)
• Assuming manual updates are needed
• Ignoring feedback from environment

2. Which of the following is the correct way to represent a self-improving agent's update step in pseudocode?

easy

A. agent.reset() every time without learning

B. agent.run() without feedback

C. agent.update(learn_from=agent.actions, feedback=environment.results)

D. agent.ignore(environment.results)

Solution

1. Step 1: Identify update step involving learning

   The agent must update itself using its actions and feedback from the environment.

2. Step 2: Match options to update logic

   Only agent.update(learn_from=agent.actions, feedback=environment.results) shows the agent updating by learning from its actions and feedback.

3. Final Answer:

   agent.update(learn_from=agent.actions, feedback=environment.results) -> Option C

4. Quick Check:

   Update with actions and feedback = A [OK]

Hint: Update means learning from actions and feedback [OK]

Common Mistakes:

• Ignoring feedback in update
• Resetting without learning
• Running without update

3. Consider this pseudocode for a self-improving agent:

actions = ['move', 'turn', 'scan']
results = [True, False, True]
agent_knowledge = {'move': 0.5, 'turn': 0.5, 'scan': 0.5}

for i in range(len(actions)):
    if results[i]:
        agent_knowledge[actions[i]] += 0.1
    else:
        agent_knowledge[actions[i]] -= 0.1

print(agent_knowledge)

What will be the printed output?

medium

A. SyntaxError

B. {'move': 0.6, 'turn': 0.4, 'scan': 0.6}

C. {'move': 0.4, 'turn': 0.6, 'scan': 0.4}

D. {'move': 0.5, 'turn': 0.5, 'scan': 0.5}

Solution

1. Step 1: Analyze loop updates on knowledge

   For each action, if result is True, add 0.1; if False, subtract 0.1.

2. Step 2: Calculate final values

   'move': 0.5 + 0.1 = 0.6; 'turn': 0.5 - 0.1 = 0.4; 'scan': 0.5 + 0.1 = 0.6.

3. Final Answer:

   {'move': 0.6, 'turn': 0.4, 'scan': 0.6} -> Option B

4. Quick Check:

   True adds 0.1, False subtracts 0.1 = D [OK]

Hint: Add 0.1 for True, subtract 0.1 for False in order [OK]

Common Mistakes:

• Not updating values correctly
• Mixing True and False effects
• Assuming no change

4. This code tries to update an agent's knowledge but has a bug:

actions = ['jump', 'run']
results = [True, False]
knowledge = {'jump': 0.3, 'run': 0.7}

for i in range(len(actions)):
    if results[i]:
        knowledge[actions[i]] += 0.1
    else:
        knowledge[actions[i]] =- 0.1

print(knowledge)

What is the bug and how to fix it?

medium

A. The operator '= -' should be '-=' to subtract; fix: change to '-='.

B. The list lengths mismatch; fix by adding more results.

C. The dictionary keys are missing; fix by adding keys.

D. The print statement is incorrect; fix by using print(knowledge.values()).

Solution

1. Step 1: Identify the incorrect operator

   The code uses '= - 0.1' which assigns negative 0.1 instead of subtracting.

2. Step 2: Correct the operator to '-='

   Changing '= -' to '-=' correctly subtracts 0.1 from the current value.

3. Final Answer:

   The operator '= -' should be '-=' to subtract; fix: change to '-='. -> Option A

4. Quick Check:

   Use '-=' to subtract, not '= -' = C [OK]

Hint: Use '-=' to subtract, not '= -' [OK]

Common Mistakes:

• Confusing '= -' with '-=' operator
• Ignoring operator syntax errors
• Thinking print statement causes error

5. You want to design a self-improving agent that adapts to changing environments by updating its strategy based on success rates. Which approach best fits this goal?

hard

A. Manually update the agent's strategy after every 100 actions.

B. Fix the agent's strategy and never update it to keep consistency.

C. Randomly change strategies without considering past results.

D. Use a feedback loop where the agent tries actions, measures success, and updates probabilities accordingly.

Solution

1. Step 1: Understand the goal of adapting strategies

   The agent must learn from success rates and update its strategy automatically.

2. Step 2: Evaluate options for self-improvement

   Only Use a feedback loop where the agent tries actions, measures success, and updates probabilities accordingly. describes a feedback loop that updates based on success, matching self-improving behavior.

3. Final Answer:

   Use a feedback loop where the agent tries actions, measures success, and updates probabilities accordingly. -> Option D

4. Quick Check:

   Feedback loop with updates = A [OK]

Hint: Use feedback loops to update strategy automatically [OK]

Common Mistakes:

• Fixing strategy without updates
• Changing randomly without feedback
• Relying on manual updates only