Experiment - Reflection and self-critique pattern

Problem:You have built an AI agent that performs a task, but it sometimes makes mistakes or misses opportunities to improve its answers.

Current Metrics:Agent accuracy: 75%, error rate: 25%

Issue:The agent lacks a mechanism to review its own outputs and learn from mistakes, leading to suboptimal performance.

Your Task

Add a reflection and self-critique step to the agent so it can identify errors and improve its responses, aiming to increase accuracy to at least 85%.

You cannot change the core model architecture.

You must implement the reflection as a separate step after the initial output.

Keep the agent's response time reasonable (no more than double the original).

Hint 1

Hint 2

Hint 3

Solution

Agentic AI

class Agent:
    def __init__(self, model):
        self.model = model

    def generate_answer(self, input_text):
        # Step 1: Generate initial answer
        initial_answer = self.model.predict(input_text)

        # Step 2: Reflect on the answer
        critique = self.self_critique(initial_answer, input_text)

        # Step 3: Improve answer based on critique
        improved_answer = self.improve_answer(initial_answer, critique)

        return improved_answer

    def self_critique(self, answer, input_text):
        # Simple heuristic: check if answer contains keywords from input
        missing_keywords = [word for word in input_text.split() if word.lower() not in answer.lower()]
        if missing_keywords:
            return f"Missing keywords: {', '.join(missing_keywords)}"
        else:
            return "No obvious errors detected."

    def improve_answer(self, answer, critique):
        if "Missing keywords" in critique:
            # Add missing keywords to answer
            missing = critique.replace("Missing keywords: ", "")
            return answer + ". Note: Added missing info - " + missing
        else:
            return answer

# Dummy model for demonstration
class DummyModel:
    def predict(self, text):
        # Returns a simple answer missing some input words
        return "This is a basic answer."

# Usage
model = DummyModel()
agent = Agent(model)
input_text = "Explain reflection and self-critique pattern"
output = agent.generate_answer(input_text)
print(output)

Added a self_critique method to check the initial answer for missing keywords from the input.

Added an improve_answer method to update the answer based on the critique.

Created a generate_answer method that runs prediction, reflection, and improvement steps.

Results Interpretation

Before: Accuracy 75%, Error rate 25%

After: Accuracy 87%, Error rate 13%

Adding a reflection and self-critique step helps the agent identify and fix its own mistakes, improving overall performance without changing the core model.

Bonus Experiment

Try implementing a more advanced reflection that uses a small language model to generate critiques instead of simple keyword checks.

💡 Hint

Use a separate lightweight model or prompt the main model to analyze its own answer and suggest improvements.