What tokens and context windows mean in AI for Everyone - Step-by-Step Execution

Concept Flow - What tokens and context windows mean

Input Text

↓

Split into Tokens

↓

Tokens Enter Context Window

↓

Model Processes Tokens

↓

Output Generated

Text is broken into tokens, which fit into a limited context window that the AI uses to understand and generate responses.

Execution Sample

AI for Everyone

Input: "Hello world!"
Tokens: ["Hello", " world", "!"]
Context window size: 4 tokens
Process tokens in window
Generate output

This example shows how input text is split into tokens, placed into a context window, then processed to produce output.

Analysis Table

Step	Action	Tokens in Context Window	Explanation
1	Receive input text	[]	No tokens yet, just raw text.
2	Split text into tokens	["Hello", " world", "!"]	Text split into 3 tokens.
3	Load tokens into context window	["Hello", " world", "!"]	All tokens fit since window size is 4.
4	Model processes tokens	["Hello", " world", "!"]	Model reads tokens to understand input.
5	Generate output	["Hello", " world", "!"]	Model uses tokens to create response.
6	End	[]	Processing complete.

💡 All tokens processed within context window size; output generated.

State Tracker

Variable	Start	After Step 2	After Step 3	After Step 5	Final
Input Text	"Hello world!"	"Hello world!"	"Hello world!"	"Hello world!"	"Hello world!"
Tokens	[]	["Hello", " world", "!"]	["Hello", " world", "!"]	["Hello", " world", "!"]	["Hello", " world", "!"]
Context Window	[]	[]	["Hello", " world", "!"]	["Hello", " world", "!"]	[]

Key Insights - 3 Insights

Why do we split text into tokens instead of using whole words?

Tokens can be smaller than words, like parts of words or punctuation, allowing the model to handle many languages and variations efficiently, as shown in step 2 of the execution_table.

What happens if the input has more tokens than the context window size?

The model can only see tokens up to the context window size at once, so older tokens may be dropped or truncated, limiting how much past text the model remembers, which is implied by the context window limit in the concept_flow.

Does the context window change during processing?

No, the context window size is fixed; tokens enter it together for processing, as shown in step 3 and 4 where all tokens fit inside the window.

Visual Quiz - 3 Questions

Look at the execution_table at step 2. How many tokens does the input text split into?

A 3 tokens

B 2 tokens

C 4 tokens

D 1 token

💡 Hint

Check the 'Tokens in Context Window' column at step 2 in the execution_table.

At which step does the model start processing the tokens?

A Step 5

B Step 3

C Step 4

D Step 2

💡 Hint

Look for the step where the action is 'Model processes tokens' in the execution_table.

If the context window size was 2 tokens instead of 4, what would happen at step 3?

A All 3 tokens still load

B Only first 2 tokens load into the context window

C No tokens load

D Tokens load one by one

💡 Hint

Context window size limits how many tokens fit at once, see concept_flow and execution_table step 3.

Concept Snapshot

Tokens are small pieces of text like words or parts of words.
A context window is the fixed number of tokens the AI can look at once.
Text is split into tokens, then loaded into the context window.
The AI processes tokens in the window to understand and respond.
If input is too long, tokens outside the window are not seen.
Context window size limits how much text the AI remembers at once.

Full Transcript

This concept explains how AI models handle text by breaking it into tokens, which are small pieces like words or parts of words. These tokens fit into a limited context window, which is the number of tokens the model can see and process at one time. The process starts with receiving input text, splitting it into tokens, loading those tokens into the context window, processing them to understand the input, and then generating output. The context window size limits how many tokens the model can consider at once, so if the input is longer than this size, some tokens may be ignored. This helps the AI manage memory and focus on recent or relevant parts of the text.

Practice

1. What does the term token mean in the context of AI language models?

easy

A. A type of AI algorithm

B. The total number of words in a document

C. The maximum length of a sentence

D. A small piece of text that AI reads one by one

Solution

1. Step 1: Understand the definition of a token

   A token is a small piece of text, like a word or part of a word, that AI processes individually.

2. Step 2: Compare options with the definition

   Only a small piece of text that AI reads one by one correctly describes a token as a small piece of text AI reads one by one.

3. Final Answer:

   A small piece of text that AI reads one by one -> Option D

4. Quick Check:

   Token = small text piece [OK]

Hint: Tokens are tiny text chunks AI reads step-by-step [OK]

Common Mistakes:

• Confusing tokens with full sentences
• Thinking tokens are AI algorithms
• Assuming tokens mean total words

2. Which of the following best describes a context window in AI language models?

easy

A. The size of the AI's training dataset

B. The maximum number of tokens AI can consider at once

C. The number of AI models running simultaneously

D. The number of layers in the AI model

Solution

1. Step 1: Define context window

   A context window is the limit on how many tokens the AI can look at together when generating or understanding text.

2. Step 2: Match definition to options

   The maximum number of tokens AI can consider at once correctly states it is the maximum number of tokens AI can consider at once.

3. Final Answer:

   The maximum number of tokens AI can consider at once -> Option B

4. Quick Check:

   Context window = max tokens at once [OK]

Hint: Context window limits tokens AI sees simultaneously [OK]

Common Mistakes:

• Mixing context window with training data size
• Confusing it with AI model layers
• Thinking it means number of AI models

3. If an AI model has a context window of 100 tokens and you input a text of 150 tokens, what happens?

medium

A. The AI ignores the first 50 tokens and processes the last 100 tokens

B. The AI processes only the first 100 tokens and ignores the rest

C. The AI processes all 150 tokens at once

D. The AI splits the text into 3 parts and processes them separately

Solution

1. Step 1: Understand context window limit

   The context window limits how many tokens the AI can consider at once, here 100 tokens.

2. Step 2: Apply limit to input length

   Since input is 150 tokens, AI can only process the last 100 tokens at once; the first 50 tokens are ignored or truncated.

3. Final Answer:

   The AI ignores the first 50 tokens and processes the last 100 tokens -> Option A

4. Quick Check:

   Context window limits tokens processed [OK]

Hint: AI processes only tokens within its context window [OK]

Common Mistakes:

• Assuming AI processes all tokens regardless of window
• Thinking AI processes first tokens only
• Believing AI splits text automatically

4. You wrote a prompt with 120 tokens, but your AI model has a context window of 100 tokens. What is a likely problem?

medium

A. The AI will produce an error or truncate the prompt

B. The AI will generate output ignoring the token limit

C. The AI will automatically increase its context window

D. The AI will repeat the prompt twice

Solution

1. Step 1: Recognize context window limits

   The AI cannot process more tokens than its context window allows, here 100 tokens.

2. Step 2: Predict AI behavior with excess tokens

   When input exceeds the limit, the AI either truncates the prompt or returns an error instead of processing all tokens.

3. Final Answer:

   The AI will produce an error or truncate the prompt -> Option A

4. Quick Check:

   Exceeding context window causes truncation or error [OK]

Hint: Too many tokens cause truncation or errors [OK]

Common Mistakes:

• Thinking AI ignores token limits
• Believing AI expands context window automatically
• Assuming AI repeats input when too long

5. You want to design an AI chatbot that remembers the last 500 tokens of conversation. The model has a context window of 300 tokens. What is a good approach?

hard

A. Split the conversation into 300-token chunks and send all at once

B. Send only the last 300 tokens each time to the AI

C. Increase the context window to 500 tokens or more

D. Ignore the context window and send all 500 tokens anyway

Solution

1. Step 1: Understand context window limits and memory needs

   The AI can only consider up to its context window tokens at once, here 300 tokens, but you want to remember 500 tokens.

2. Step 2: Find solution to match memory with context window

   To remember 500 tokens, the context window must be at least 500 tokens; increasing it is the correct approach.

3. Final Answer:

   Increase the context window to 500 tokens or more -> Option C

4. Quick Check:

   Context window must fit tokens to remember [OK]

Hint: Context window must be large enough to hold all tokens [OK]

Common Mistakes:

• Sending more tokens than context window allows
• Trying to send multiple chunks at once
• Ignoring context window limits