Context window and token limits in Prompt Engineering / GenAI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding context window size

If a language model has a context window of 2048 tokens, what happens when you input a text longer than 2048 tokens?

A The model processes only the last 2048 tokens, ignoring the earlier ones.

B The model processes all tokens by splitting them into multiple windows automatically.

C The model raises an error and refuses to process the input.

D The model processes only the first 2048 tokens and ignores the rest.

💡 Hint

Think about how models handle inputs that exceed their maximum token capacity.

Language models with fixed context windows can only consider a limited number of tokens at once. When input exceeds this limit, the model typically uses the most recent tokens within the window, ignoring earlier tokens.

❓ Predict Output

intermediate

Token count calculation

Given the following Python code using the Hugging Face tokenizer, what is the output of the print statement?

Prompt Engineering / GenAI

from transformers import GPT2Tokenizer

tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
text = 'Hello world! This is a test.'
tokens = tokenizer.encode(text)
print(len(tokens))

A 5

B 8

C 6

D 7

💡 Hint

Count how the tokenizer splits the sentence into tokens.

The GPT-2 tokenizer splits 'Hello world! This is a test.' into 7 tokens: ['Hello', 'Ġworld', '!', 'ĠThis', 'Ġis', 'Ġa', 'Ġtest', '.'] is incorrect because the final '.' is part of the last token 'Ġtest.'.

❓ Hyperparameter

advanced

Choosing context window size for training

When training a transformer model, increasing the context window size from 512 to 2048 tokens will most likely:

A Increase memory usage and training time significantly.

B Decrease the model's ability to understand long texts.

C Reduce the number of model parameters.

D Make the model train faster due to fewer batches.

💡 Hint

Think about how longer sequences affect computation in transformers.

Longer context windows mean the model processes more tokens at once, increasing memory and computation time, which slows training and requires more resources.

❓ Metrics

advanced

Effect of token limits on model evaluation

If a model's context window is 1024 tokens but the evaluation dataset contains samples of 1500 tokens, what is the likely effect on the evaluation metrics?

A Metrics improve because longer inputs provide more information.

B Metrics may be worse because the model cannot see the full input context.

C Metrics remain unchanged as the model truncates inputs automatically without impact.

D Metrics become invalid because the model crashes on long inputs.

💡 Hint

Consider how truncating input affects model understanding.

When inputs exceed the context window, the model only sees part of the input, which can reduce its ability to make accurate predictions, hurting evaluation metrics.

🔧 Debug

expert

Diagnosing token limit errors in generation

You use a language model with a 2048 token limit. Your code generates text by appending new tokens to the input prompt repeatedly. After some iterations, generation fails with a token limit error. What is the best way to fix this?

A Increase the model's token limit by changing a parameter in the code.

B Restart generation from scratch every time to avoid token buildup.

C Truncate the oldest tokens from the prompt to keep total tokens under 2048.

D Ignore the error and continue generating tokens beyond the limit.

💡 Hint

Think about how to keep the input size manageable during generation.

Since the model cannot process more than 2048 tokens, removing the oldest tokens keeps the input within limits while preserving recent context for generation.