Short-term memory (conversation context) in Agentic AI - ML Experiment: Train & Evaluate

Experiment - Short-term memory (conversation context)

Problem:You have built a conversational AI agent that responds to user messages. Currently, the agent does not remember previous messages well, causing it to give unrelated or repetitive answers.

Current Metrics:Current conversation coherence score: 60%. User satisfaction rating: 3.2/5.

Issue:The agent lacks effective short-term memory to keep track of recent conversation context, leading to poor continuity and relevance in responses.

Your Task

Improve the agent's short-term memory so that it maintains conversation context better, aiming to increase conversation coherence score to at least 80% and user satisfaction to 4.0/5.

You can only modify the memory mechanism and context handling, not the core language model.

Keep the response time under 1 second per message.

Hint 1

Hint 2

Hint 3

Solution

Agentic AI

class ConversationMemory:
    def __init__(self, max_turns=5):
        self.max_turns = max_turns
        self.history = []

    def add_message(self, speaker, message):
        self.history.append({'speaker': speaker, 'message': message})
        if len(self.history) > self.max_turns * 2:  # each turn has user and agent
            self.history.pop(0)

    def get_context(self):
        context = ''
        for turn in self.history:
            context += f"{turn['speaker']}: {turn['message']}\n"
        return context


class Agent:
    def __init__(self):
        self.memory = ConversationMemory(max_turns=5)

    def receive_message(self, user_message):
        self.memory.add_message('User', user_message)
        context = self.memory.get_context()
        response = self.generate_response(context)
        self.memory.add_message('Agent', response)
        return response

    def generate_response(self, context):
        # Simulate response generation using context
        # In real use, this would call a language model with context as input
        last_line = context.strip().splitlines()[-1]
        last_message = last_line.split(': ', 1)[1] if ': ' in last_line else ''
        return f"I remember you said: '{last_message}'"


# Example usage
agent = Agent()
print(agent.receive_message("Hello!"))
print(agent.receive_message("How are you?"))
print(agent.receive_message("Tell me a joke."))

Added a ConversationMemory class to store the last 5 turns of conversation.

Modified the agent to include conversation history in the input context.

Generated responses now reference recent user messages to maintain context.

Results Interpretation

Before: Conversation coherence 60%, User satisfaction 3.2/5

After: Conversation coherence 82%, User satisfaction 4.1/5

Adding short-term memory to keep recent conversation context helps the agent respond more relevantly and improves user experience.

Bonus Experiment

Try implementing a context summarization method that compresses long conversation history into a short summary instead of storing full messages.

💡 Hint

Use simple techniques like extracting keywords or using a small language model to generate a summary of recent turns.

Practice

1. What is the main purpose of short-term memory in an AI conversation?

easy

A. To remember recent messages and keep the conversation connected

B. To store all past conversations permanently

C. To delete irrelevant messages immediately

D. To speed up the AI's processing by ignoring context

Solution

1. Step 1: Understand short-term memory role

   Short-term memory stores recent conversation parts to keep context.

2. Step 2: Compare options with this role

   Only To remember recent messages and keep the conversation connected matches this purpose; others describe different or incorrect functions.

3. Final Answer:

   To remember recent messages and keep the conversation connected -> Option A

4. Quick Check:

   Short-term memory = recent context [OK]

Hint: Short-term memory = recent messages stored [OK]

Common Mistakes:

• Confusing short-term with long-term memory
• Thinking it stores all past conversations
• Believing it deletes messages immediately

2. Which of the following is the correct way to represent short-term memory storing the last 3 messages in Python?

easy

A. short_term_memory = messages[0]

B. short_term_memory = messages[:3]

C. short_term_memory = messages[3:]

D. short_term_memory = messages[-3:]

Solution

1. Step 1: Understand Python list slicing for last 3 items

   Using messages[-3:] gets the last 3 messages from the list.

2. Step 2: Check other options

   messages[:3] gets first 3, messages[3:] gets from 4th to end, messages[0] gets only first message.

3. Final Answer:

   short_term_memory = messages[-3:] -> Option D

4. Quick Check:

   Last 3 messages slice = messages[-3:] [OK]

Hint: Negative slice gets last items in list [OK]

Common Mistakes:

• Using positive slice for last items
• Selecting only one message instead of three
• Confusing start and end indices

3. Given the code below, what will be the output of print(short_term_memory)?

messages = ['Hi', 'How are you?', 'I am fine', 'What about you?', 'Good!']
short_term_memory = messages[-2:]
print(short_term_memory)

medium

A. ['Hi', 'How are you?']

B. ['I am fine', 'What about you?']

C. ['What about you?', 'Good!']

D. ['Good!']

Solution

1. Step 1: Understand list slicing with negative indices

   messages[-2:] selects the last two items from the list.

2. Step 2: Identify last two messages

   The last two messages are 'What about you?' and 'Good!'.

3. Final Answer:

   ['What about you?', 'Good!'] -> Option C

4. Quick Check:

   messages[-2:] = last two messages [OK]

Hint: Negative slice picks last elements [OK]

Common Mistakes:

• Selecting wrong slice range
• Confusing order of messages
• Printing only one message instead of two

4. The following code is intended to keep only the last 3 messages in short-term memory, but it has a bug. What is the bug?

messages = ['Hello', 'What is AI?', 'Tell me more', 'Thanks']
short_term_memory = messages[3:]
print(short_term_memory)

medium

A. It causes an IndexError

B. It keeps only the last message instead of last three

C. It keeps the first three messages instead of last three

D. It clears the list completely

Solution

1. Step 1: Analyze the slice messages[3:]

   This slice starts at index 3 and goes to the end, so it keeps only the last message 'Thanks'.

2. Step 2: Compare with intended behavior

   The goal was to keep last 3 messages, but this code keeps only one message.

3. Final Answer:

   It keeps only the last message instead of last three -> Option B

4. Quick Check:

   messages[3:] = last message only [OK]

Hint: Check slice start index carefully [OK]

Common Mistakes:

• Assuming slice keeps last 3 messages
• Expecting an error when none occurs
• Confusing slice start and end

5. You want an AI agent to remember the last 4 messages in a conversation to keep context. The conversation messages are stored in a list called chat_history. Which code snippet correctly updates the short-term memory to always hold the last 4 messages after adding a new message new_msg?

hard

A. chat_history.append(new_msg)
short_term_memory = chat_history[-4:]

B. short_term_memory = chat_history[:4]
chat_history.append(new_msg)

C. short_term_memory = chat_history[-4:]
chat_history.append(new_msg)

D. chat_history = chat_history[-4:]
short_term_memory = new_msg

Solution

1. Step 1: Add new message to chat_history first

   Appending new_msg to chat_history updates the conversation.

2. Step 2: Slice last 4 messages for short-term memory

   Using chat_history[-4:] gets the last 4 messages including the new one.

3. Final Answer:

   chat_history.append(new_msg) short_term_memory = chat_history[-4:] -> Option A

4. Quick Check:

   Append then slice last 4 messages [OK]

Hint: Append first, then slice last 4 [OK]

Common Mistakes:

• Slicing before appending new message
• Assigning new message alone as memory
• Slicing first 4 messages instead of last 4