Memory for conversation history in Prompt Engineering / GenAI - Model Metrics & Evaluation

Metrics & Evaluation - Memory for conversation history

Which metric matters for Memory for conversation history and WHY

When a model remembers past conversation, we want to check how well it keeps important details without mixing up or forgetting. Key metrics include Recall to see if the model remembers all relevant past info, and Precision to ensure it doesn't add wrong or unrelated info. Also, F1 score balances these two. These metrics help us know if the memory is accurate and complete, which is vital for smooth, meaningful chats.

Confusion matrix for Memory recall and precision

      |-----------------------------|
      |          | Predicted Memory |
      | Actual   | Relevant | Wrong  |
      | Memory   |          |        |
      |-----------------------------|
      | Relevant |   TP     |   FP   |
      | Wrong    |   FN     |   TN   |
      |-----------------------------|

      TP = Correctly remembered important info
      FP = Incorrect or unrelated info remembered
      FN = Important info forgotten
      TN = Correctly ignored irrelevant info
    

Precision vs Recall tradeoff with examples

High Precision, Low Recall: The model remembers only very sure facts, avoiding mistakes but forgetting some details. Good if wrong info is harmful.

High Recall, Low Precision: The model tries to remember everything, including some wrong or irrelevant details. Good if missing info is worse than some mistakes.

For example, in a customer support chat, high recall helps remember all user issues, but high precision avoids confusing the user with wrong info.

What "good" vs "bad" metric values look like for Memory

Good: Precision and Recall both above 0.8 means the model remembers most important info and rarely adds wrong details.

Bad: Precision below 0.5 means many wrong memories; Recall below 0.5 means many forgotten details. Either harms conversation quality.

Common pitfalls in evaluating Memory metrics

• Accuracy paradox: High overall accuracy can hide poor memory if irrelevant info dominates.
• Data leakage: If test data leaks past conversation, metrics look better but don't reflect real memory ability.
• Overfitting: Model may memorize training chats perfectly but fail on new conversations.

Self-check question

Your chat model has 98% accuracy remembering conversation history but only 12% recall on important past details. Is it good for real use? Why or why not?

Answer: No, it is not good. The low recall means the model forgets most important info, even if overall accuracy looks high. This will cause poor chat quality because key details are missed.

Key Result

For memory in conversation, balance high recall and precision to keep important details without adding wrong info.

Practice

1. What is the main purpose of memory in a conversation AI system?

easy

A. To store past user and AI messages for context

B. To speed up the internet connection

C. To generate random responses without context

D. To delete all previous messages after each reply

Solution

1. Step 1: Understand the role of memory in AI conversations

   Memory keeps track of previous messages so the AI can understand the flow of the conversation.

2. Step 2: Identify the correct purpose

   Storing past messages helps the AI respond with context, making conversations meaningful.

3. Final Answer:

   To store past user and AI messages for context -> Option A

4. Quick Check:

   Memory = store past messages [OK]

Hint: Memory keeps conversation context, not random or deleted [OK]

Common Mistakes:

• Thinking memory speeds up internet
• Believing memory deletes all messages
• Assuming memory generates random replies

2. Which of the following is the correct way to add a new message to conversation memory in Python?

easy

A. memory.append(new_message)

B. memory.add(new_message)

C. memory.insert(new_message)

D. memory.push(new_message)

Solution

1. Step 1: Recall Python list methods for adding items

   Python lists use append() to add an item at the end.

2. Step 2: Match method to memory update

   Since conversation memory is often a list, append() is the correct method to add a new message.

3. Final Answer:

   memory.append(new_message) -> Option A

4. Quick Check:

   Python list add = append() [OK]

Hint: Use append() to add items to a Python list [OK]

Common Mistakes:

• Using add() which is for sets
• Using insert() without index
• Using push() which is not a Python list method

3. Given this Python code snippet managing conversation memory:

memory = ['Hi', 'How are you?']
new_message = 'I am fine'
memory.append(new_message)
print(len(memory))

What will be the output?

medium

A. 2

B. 3

C. 1

D. Error

Solution

1. Step 1: Check initial memory length

   Memory starts with 2 messages: 'Hi' and 'How are you?'.

2. Step 2: Append new message and count

   Appending 'I am fine' adds one more message, so total becomes 3.

3. Final Answer:

   3 -> Option B

4. Quick Check:

   2 + 1 = 3 messages [OK]

Hint: Appending adds one item, so length increases by 1 [OK]

Common Mistakes:

• Forgetting append adds item
• Thinking length stays same
• Assuming code causes error

4. You have this code to keep conversation memory but it causes an error:

memory = []
new_message = 'Hello'
memory.add(new_message)

What is the error and how to fix it?

medium

A. No error; code runs fine

B. Error: new_message undefined; fix by defining new_message

C. Error: list has no add(); fix by using memory.append(new_message)

D. Error: memory is not a list; fix by initializing memory as a dict

Solution

1. Step 1: Identify the error cause

   Python lists do not have an add() method; this causes an AttributeError.

2. Step 2: Correct method to add item to list

   Use append() to add an item to a list, so replace add() with append().

3. Final Answer:

   Error: list has no add(); fix by using memory.append(new_message) -> Option C

4. Quick Check:

   List add() wrong, use append() [OK]

Hint: Lists use append(), sets use add() [OK]

Common Mistakes:

• Using add() on list
• Thinking new_message is undefined
• Confusing list with dict

5. You want to keep only the last 3 messages in conversation memory to save space. Which code correctly updates memory after adding a new message?

hard

A. memory.insert(0, new_message)
memory = memory[:3]

B. memory = memory[:3]
memory.append(new_message)

C. memory.pop()
memory.append(new_message)

D. memory.append(new_message)
memory = memory[-3:]

Solution

1. Step 1: Add new message to memory

   Use append() to add the new message at the end.

2. Step 2: Keep only last 3 messages

   Slicing with memory[-3:] keeps the last 3 items, removing older ones.

3. Final Answer:

   memory.append(new_message) memory = memory[-3:] -> Option D

4. Quick Check:

   Append then slice last 3 [OK]

Hint: Append first, then slice last 3 messages [OK]

Common Mistakes:

• Slicing before append loses new message
• Using insert at start changes order
• Popping removes wrong message