Memory for conversation history in Prompt Engineering / GenAI - Model Pipeline Trace

Model Pipeline - Memory for conversation history

This pipeline shows how a conversation AI remembers past messages to give better answers. It stores and updates conversation history, then uses it to understand new questions and respond well.

Data Flow - 6 Stages

1User Input

1 message string→Receive new user message→1 message string

"What is the weather today?"

↓

2Retrieve Conversation History

Previous conversation messages (5 messages x 1 string each)→Load stored past messages→5 messages x 1 string each

["Hi!", "Hello, how can I help?", "Tell me a joke.", "Why did the chicken cross the road?", "To get to the other side."]

↓

3Combine Input and History

1 new message + 5 past messages→Concatenate messages into one context→1 combined text string

"Hi! Hello, how can I help? Tell me a joke. Why did the chicken cross the road? To get to the other side. What is the weather today?"

↓

4Encode Context

1 combined text string→Convert text to numerical vectors→1 vector of length 768

[0.12, -0.05, 0.33, ..., 0.07]

↓

5Generate Response

1 vector of length 768→Model predicts next message tokens→1 response string

"The weather today is sunny with a high of 75 degrees."

↓

6Update Conversation History

Previous 5 messages + new user message + new response→Add new messages and remove oldest if needed→Updated 5 messages x 1 string each

["Tell me a joke.", "Why did the chicken cross the road?", "To get to the other side.", "What is the weather today?", "The weather today is sunny with a high of 75 degrees."]

Training Trace - Epoch by Epoch

Loss
1.2 |*       
1.0 | **     
0.8 |  ***   
0.6 |   **** 
0.4 |    *****
     --------
     Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.45	Model starts learning to remember conversation context.
2	0.9	0.60	Loss decreases as model better understands history.
3	0.7	0.72	Model improves at generating relevant responses.
4	0.5	0.80	Training converges; model remembers longer history.
5	0.4	0.85	Final epoch with good balance of memory and response quality.

Prediction Trace - 6 Layers

Layer 1: Input message

Layer 2: Retrieve conversation history

Layer 3: Combine input and history

Layer 4: Encode context

Layer 5: Generate response

Layer 6: Update conversation history

Model Quiz - 3 Questions

What happens to the conversation history after a new response is generated?

A It stays the same without changes.

B It is updated by adding new messages and removing oldest ones.

C It is deleted completely.

D It is converted into images.

Key Insight

Remembering past conversation messages helps the AI give relevant and coherent answers. The model learns to store and update history, improving response quality over time.

Practice

1. What is the main purpose of memory in a conversation AI system?

easy

A. To store past user and AI messages for context

B. To speed up the internet connection

C. To generate random responses without context

D. To delete all previous messages after each reply

Solution

1. Step 1: Understand the role of memory in AI conversations

   Memory keeps track of previous messages so the AI can understand the flow of the conversation.

2. Step 2: Identify the correct purpose

   Storing past messages helps the AI respond with context, making conversations meaningful.

3. Final Answer:

   To store past user and AI messages for context -> Option A

4. Quick Check:

   Memory = store past messages [OK]

Hint: Memory keeps conversation context, not random or deleted [OK]

Common Mistakes:

• Thinking memory speeds up internet
• Believing memory deletes all messages
• Assuming memory generates random replies

2. Which of the following is the correct way to add a new message to conversation memory in Python?

easy

A. memory.append(new_message)

B. memory.add(new_message)

C. memory.insert(new_message)

D. memory.push(new_message)

Solution

1. Step 1: Recall Python list methods for adding items

   Python lists use append() to add an item at the end.

2. Step 2: Match method to memory update

   Since conversation memory is often a list, append() is the correct method to add a new message.

3. Final Answer:

   memory.append(new_message) -> Option A

4. Quick Check:

   Python list add = append() [OK]

Hint: Use append() to add items to a Python list [OK]

Common Mistakes:

• Using add() which is for sets
• Using insert() without index
• Using push() which is not a Python list method

3. Given this Python code snippet managing conversation memory:

memory = ['Hi', 'How are you?']
new_message = 'I am fine'
memory.append(new_message)
print(len(memory))

What will be the output?

medium

A. 2

B. 3

C. 1

D. Error

Solution

1. Step 1: Check initial memory length

   Memory starts with 2 messages: 'Hi' and 'How are you?'.

2. Step 2: Append new message and count

   Appending 'I am fine' adds one more message, so total becomes 3.

3. Final Answer:

   3 -> Option B

4. Quick Check:

   2 + 1 = 3 messages [OK]

Hint: Appending adds one item, so length increases by 1 [OK]

Common Mistakes:

• Forgetting append adds item
• Thinking length stays same
• Assuming code causes error

4. You have this code to keep conversation memory but it causes an error:

memory = []
new_message = 'Hello'
memory.add(new_message)

What is the error and how to fix it?

medium

A. No error; code runs fine

B. Error: new_message undefined; fix by defining new_message

C. Error: list has no add(); fix by using memory.append(new_message)

D. Error: memory is not a list; fix by initializing memory as a dict

Solution

1. Step 1: Identify the error cause

   Python lists do not have an add() method; this causes an AttributeError.

2. Step 2: Correct method to add item to list

   Use append() to add an item to a list, so replace add() with append().

3. Final Answer:

   Error: list has no add(); fix by using memory.append(new_message) -> Option C

4. Quick Check:

   List add() wrong, use append() [OK]

Hint: Lists use append(), sets use add() [OK]

Common Mistakes:

• Using add() on list
• Thinking new_message is undefined
• Confusing list with dict

5. You want to keep only the last 3 messages in conversation memory to save space. Which code correctly updates memory after adding a new message?

hard

A. memory.insert(0, new_message)
memory = memory[:3]

B. memory = memory[:3]
memory.append(new_message)

C. memory.pop()
memory.append(new_message)

D. memory.append(new_message)
memory = memory[-3:]

Solution

1. Step 1: Add new message to memory

   Use append() to add the new message at the end.

2. Step 2: Keep only last 3 messages

   Slicing with memory[-3:] keeps the last 3 items, removing older ones.

3. Final Answer:

   memory.append(new_message) memory = memory[-3:] -> Option D

4. Quick Check:

   Append then slice last 3 [OK]

Hint: Append first, then slice last 3 messages [OK]

Common Mistakes:

• Slicing before append loses new message
• Using insert at start changes order
• Popping removes wrong message