What tokens and context windows mean in AI for Everyone - Mini Project: Build & Apply

Understanding Tokens and Context Windows

📖 Scenario: You are learning how AI language models understand and process text. To do this, you need to understand two key ideas: tokens and context windows.Imagine you are reading a book, but instead of reading whole sentences at once, you read small pieces called tokens. Also, you can only remember a certain number of these tokens at a time, which is called the context window.

🎯 Goal: Build a simple explanation using examples to show what tokens are and how a context window limits what an AI can remember at once.

📋 What You'll Learn

Create a list of example tokens from a sentence

Define a variable for the context window size

Select tokens that fit inside the context window

Explain the final selection as the AI's current memory

💡 Why This Matters

🌍 Real World

Understanding tokens and context windows helps people grasp how AI models read and remember text, which is important for using AI tools effectively.

💼 Career

This knowledge is useful for anyone working with AI, such as developers, data scientists, or content creators, to better design and interact with AI systems.

1

Create a list of tokens from a sentence

Create a list called tokens with these exact string elements: 'I', 'love', 'learning', 'about', 'AI'.

AI for Everyone

# Create the list of tokens
# Your code here

Need a hint?

Think of tokens as small pieces of a sentence, like words.

2

Define the context window size

Create a variable called context_window and set it to the number 3.

AI for Everyone

tokens = ['I', 'love', 'learning', 'about', 'AI']
# Define the context window size
# Your code here

Need a hint?

The context window is how many tokens the AI can remember at once.

3

Select tokens within the context window

Create a new list called current_context that contains the first context_window tokens from the tokens list.

AI for Everyone

tokens = ['I', 'love', 'learning', 'about', 'AI']
context_window = 3
# Select tokens within the context window
# Your code here

Need a hint?

Use slicing to get the first few tokens that fit in the context window.

4

Explain the AI's current memory

Add a comment explaining that current_context represents the tokens the AI can remember at once within its context window.

AI for Everyone

tokens = ['I', 'love', 'learning', 'about', 'AI']
context_window = 3
current_context = tokens[:context_window]
# Explain the AI's current memory
# Your code here

Need a hint?

Write a clear comment about what current_context means for AI memory.