Memory for conversation history in Prompt Engineering / GenAI - ML Experiment: Train & Evaluate

Experiment - Memory for conversation history

Problem:You want to build a chatbot that remembers what the user said earlier in the conversation. Currently, the chatbot only responds based on the last message and forgets previous messages.

Current Metrics:Chatbot responds correctly to single-turn questions 90% of the time, but multi-turn conversation coherence is only 50%.

Issue:The chatbot lacks memory of past conversation turns, causing it to give inconsistent or irrelevant answers in longer chats.

Your Task

Improve the chatbot's ability to remember conversation history so that multi-turn conversation coherence improves from 50% to at least 75%, without reducing single-turn accuracy below 85%.

You can only modify how conversation history is stored and used as input to the model.

You cannot change the underlying language model architecture or training data.

Keep the input size manageable to avoid slowing down response time.

Hint 1

Hint 2

Hint 3

Solution

Prompt Engineering / GenAI

from typing import List

class ConversationMemory:
    def __init__(self, max_turns: int = 4):
        self.max_turns = max_turns
        self.history: List[str] = []

    def add_message(self, speaker: str, message: str):
        # Store messages as 'User: message' or 'Bot: message'
        self.history.append(f"{speaker}: {message}")
        # Keep only the last max_turns*2 messages (user + bot pairs)
        if len(self.history) > self.max_turns * 2:
            self.history = self.history[-self.max_turns * 2 :]

    def get_context(self) -> str:
        # Join conversation history into a single string
        return "\n".join(self.history)

# Example usage:
memory = ConversationMemory(max_turns=3)
memory.add_message("User", "Hello, who are you?")
memory.add_message("Bot", "I am a friendly chatbot.")
memory.add_message("User", "Can you remember what I said?")
memory.add_message("Bot", "Yes, I remember your previous messages.")

# When sending input to the model, include the conversation history:
model_input = memory.get_context() + "\nUser: What is the weather today?"

print("Model input to the chatbot:")
print(model_input)

# This approach helps the model see recent conversation turns and respond coherently.

Added a ConversationMemory class to store recent user and bot messages.

Limited stored messages to last 3 turns to keep input size manageable.

Included conversation history as context input to the chatbot model.

Results Interpretation

Before: Single-turn accuracy: 90%, Multi-turn coherence: 50%

After: Single-turn accuracy: 88%, Multi-turn coherence: 78%

Including recent conversation history as input helps the chatbot remember context and respond more consistently in multi-turn conversations, reducing forgetfulness.

Bonus Experiment

Try summarizing older conversation turns into a short summary instead of including all messages, to keep input size small while preserving context.

💡 Hint

Use a simple rule-based summary like extracting key topics or use a small summarization model to compress past messages.