T5 for text-to-text tasks in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the main advantage of T5's text-to-text framework?

T5 treats every problem as a text-to-text task. What is the main advantage of this approach?

A It allows using the same model architecture and training procedure for many different NLP tasks.

B It requires separate models for each task to improve accuracy.

C It only works for translation tasks and not for classification.

D It eliminates the need for any pre-training on large datasets.

💡 Hint

Think about how treating all tasks as text input and output simplifies the model design.

T5's text-to-text framework means the same model can be used for translation, summarization, classification, and more by just changing the input and output text format.

❓ Predict Output

intermediate

Output of T5 model generating summary

Given the input text: "summarize: The cat sat on the mat and looked outside." What is the most likely output of a T5 model fine-tuned for summarization?

NLP

from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained('t5-small')
model = T5ForConditionalGeneration.from_pretrained('t5-small')

input_text = 'summarize: The cat sat on the mat and looked outside.'
input_ids = tokenizer(input_text, return_tensors='pt').input_ids

outputs = model.generate(input_ids, max_length=10)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

A "The mat was sat on by the cat."

B "The cat sat on the mat."

C "The cat sat on the mat and looked outside."

D "The cat looked outside."

💡 Hint

Summarization usually shortens and keeps the main idea.

The T5 model fine-tuned for summarization will produce a shorter sentence capturing the main idea, here "The cat looked outside." is the best summary.

❓ Hyperparameter

advanced

Choosing max_length for T5 text generation

When generating text with T5, which max_length value is best to avoid cutting off important output while keeping generation efficient?

A Set max_length to a very large number like 1000 to ensure full output.

B Set max_length to a small number like 5 to speed up generation.

C Set max_length to a reasonable length based on expected output size, e.g., 50 for summaries.

D Do not set max_length; let the model generate indefinitely.

💡 Hint

Think about balancing output completeness and speed.

Choosing max_length based on expected output size avoids cutting off important text and keeps generation efficient.

❓ Metrics

advanced

Evaluating T5 model performance on summarization

Which metric is most appropriate to evaluate the quality of summaries generated by a T5 model?

A BLEU score - measures overlap of n-grams between generated and reference text.

B Accuracy - percentage of exact matches with reference summaries.

C Mean Squared Error - difference between predicted and actual token IDs.

D Perplexity - measures how well the model predicts the next token.

💡 Hint

Think about metrics that compare generated text to reference text.

BLEU score is commonly used to evaluate text generation tasks like summarization by measuring n-gram overlap.

🔧 Debug

expert

Why does T5 generate repetitive text?

You fine-tuned a T5 model for text generation, but the output repeats the same phrase multiple times. What is the most likely cause?

A The model's max_length is too short, causing repetition.

B The decoding strategy lacks diversity, e.g., greedy decoding without penalties.

C The beam search size is too large, causing repeated phrases.

D The tokenizer vocabulary is too small, causing repetition.

💡 Hint

Consider how decoding methods affect output variety.

Greedy decoding or beam search without diversity penalties can cause repetitive outputs in text generation.