Short-term memory (conversation context) in Agentic AI - Model Pipeline Trace

Model Pipeline - Short-term memory (conversation context)

This pipeline shows how an AI agent keeps track of recent conversation to understand and respond better. It stores recent messages, processes them, and uses this memory to predict the next reply.

Data Flow - 5 Stages

1Input conversation

1 conversation with 5 recent messages→Collect recent messages as text strings→5 messages as text array

["Hi!", "How are you?", "What's the weather?", "It's sunny.", "Great!"]

↓

2Preprocessing

5 messages as text array→Convert text to token IDs using tokenizer→5 sequences of token IDs

[[101, 7632, 999], [101, 2129, 2024, 2017, 1029], ...]

↓

3Short-term memory encoding

5 sequences of token IDs→Encode sequences into fixed-size memory vectors→5 vectors of size 128

[[0.12, -0.05, ...], [0.08, 0.11, ...], ...]

↓

4Memory aggregation

5 vectors of size 128→Combine vectors into one context vector→1 vector of size 128

[0.10, 0.02, -0.01, ...]

↓

5Next response prediction

1 context vector of size 128→Use context vector to predict next reply tokens→Sequence of predicted token IDs

[101, 2204, 2017, 2064, 2173, 102]

Training Trace - Epoch by Epoch

1.2 |***************
1.0 |************
0.8 |**********
0.6 |*******
0.4 |****
    +----------------
     1  2  3  4  5  epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.20	0.45	Model starts learning conversation patterns
2	0.95	0.60	Loss decreases, accuracy improves
3	0.75	0.70	Model better understands short-term context
4	0.60	0.78	Continued improvement in prediction
5	0.50	0.83	Model converges with good accuracy

Prediction Trace - 5 Layers

Layer 1: Input recent messages

Layer 2: Tokenization

Layer 3: Encoding messages

Layer 4: Aggregate memory

Layer 5: Predict next reply tokens

Model Quiz - 3 Questions

What does the short-term memory encoding stage do?

A Combines all vectors into one

B Turns text messages into vectors

C Predicts the next reply tokens

D Collects recent messages

Key Insight

Short-term memory helps AI agents remember recent conversation parts as vectors. This memory guides better predictions for the next reply, improving with training as loss decreases and accuracy rises.

Practice

1. What is the main purpose of short-term memory in an AI conversation?

easy

A. To remember recent messages and keep the conversation connected

B. To store all past conversations permanently

C. To delete irrelevant messages immediately

D. To speed up the AI's processing by ignoring context

Solution

1. Step 1: Understand short-term memory role

   Short-term memory stores recent conversation parts to keep context.

2. Step 2: Compare options with this role

   Only To remember recent messages and keep the conversation connected matches this purpose; others describe different or incorrect functions.

3. Final Answer:

   To remember recent messages and keep the conversation connected -> Option A

4. Quick Check:

   Short-term memory = recent context [OK]

Hint: Short-term memory = recent messages stored [OK]

Common Mistakes:

• Confusing short-term with long-term memory
• Thinking it stores all past conversations
• Believing it deletes messages immediately

2. Which of the following is the correct way to represent short-term memory storing the last 3 messages in Python?

easy

A. short_term_memory = messages[0]

B. short_term_memory = messages[:3]

C. short_term_memory = messages[3:]

D. short_term_memory = messages[-3:]

Solution

1. Step 1: Understand Python list slicing for last 3 items

   Using messages[-3:] gets the last 3 messages from the list.

2. Step 2: Check other options

   messages[:3] gets first 3, messages[3:] gets from 4th to end, messages[0] gets only first message.

3. Final Answer:

   short_term_memory = messages[-3:] -> Option D

4. Quick Check:

   Last 3 messages slice = messages[-3:] [OK]

Hint: Negative slice gets last items in list [OK]

Common Mistakes:

• Using positive slice for last items
• Selecting only one message instead of three
• Confusing start and end indices

3. Given the code below, what will be the output of print(short_term_memory)?

messages = ['Hi', 'How are you?', 'I am fine', 'What about you?', 'Good!']
short_term_memory = messages[-2:]
print(short_term_memory)

medium

A. ['Hi', 'How are you?']

B. ['I am fine', 'What about you?']

C. ['What about you?', 'Good!']

D. ['Good!']

Solution

1. Step 1: Understand list slicing with negative indices

   messages[-2:] selects the last two items from the list.

2. Step 2: Identify last two messages

   The last two messages are 'What about you?' and 'Good!'.

3. Final Answer:

   ['What about you?', 'Good!'] -> Option C

4. Quick Check:

   messages[-2:] = last two messages [OK]

Hint: Negative slice picks last elements [OK]

Common Mistakes:

• Selecting wrong slice range
• Confusing order of messages
• Printing only one message instead of two

4. The following code is intended to keep only the last 3 messages in short-term memory, but it has a bug. What is the bug?

messages = ['Hello', 'What is AI?', 'Tell me more', 'Thanks']
short_term_memory = messages[3:]
print(short_term_memory)

medium

A. It causes an IndexError

B. It keeps only the last message instead of last three

C. It keeps the first three messages instead of last three

D. It clears the list completely

Solution

1. Step 1: Analyze the slice messages[3:]

   This slice starts at index 3 and goes to the end, so it keeps only the last message 'Thanks'.

2. Step 2: Compare with intended behavior

   The goal was to keep last 3 messages, but this code keeps only one message.

3. Final Answer:

   It keeps only the last message instead of last three -> Option B

4. Quick Check:

   messages[3:] = last message only [OK]

Hint: Check slice start index carefully [OK]

Common Mistakes:

• Assuming slice keeps last 3 messages
• Expecting an error when none occurs
• Confusing slice start and end

5. You want an AI agent to remember the last 4 messages in a conversation to keep context. The conversation messages are stored in a list called chat_history. Which code snippet correctly updates the short-term memory to always hold the last 4 messages after adding a new message new_msg?

hard

A. chat_history.append(new_msg)
short_term_memory = chat_history[-4:]

B. short_term_memory = chat_history[:4]
chat_history.append(new_msg)

C. short_term_memory = chat_history[-4:]
chat_history.append(new_msg)

D. chat_history = chat_history[-4:]
short_term_memory = new_msg

Solution

1. Step 1: Add new message to chat_history first

   Appending new_msg to chat_history updates the conversation.

2. Step 2: Slice last 4 messages for short-term memory

   Using chat_history[-4:] gets the last 4 messages including the new one.

3. Final Answer:

   chat_history.append(new_msg) short_term_memory = chat_history[-4:] -> Option A

4. Quick Check:

   Append then slice last 4 messages [OK]

Hint: Append first, then slice last 4 [OK]

Common Mistakes:

• Slicing before appending new message
• Assigning new message alone as memory
• Slicing first 4 messages instead of last 4