Why text generation creates content in NLP - Challenge Your Understanding

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why does text generation produce new content?

Text generation models create new sentences instead of copying existing ones. Why is this possible?

A Because the model stores all sentences from training and randomly selects one to output.

B Because the model learns patterns and relationships in language, allowing it to predict and create new sequences.

C Because the model uses a fixed dictionary of phrases and repeats them in order.

D Because the model only copies the first sentence it saw during training.

💡 Hint

Think about how the model understands language structure rather than memorizing exact sentences.

Text generation models learn the rules and patterns of language from data. They use this knowledge to predict the next word in a sequence, which lets them create new, unique sentences rather than just repeating what they saw.

❓ Predict Output

intermediate

Output of a simple text generation step

Given a trained language model that predicts the next word, what is the output of this code snippet?

NLP

import random

vocab = ['hello', 'world', 'machine', 'learning']

# Simulate next word prediction probabilities
probs = [0.1, 0.7, 0.1, 0.1]

next_word = random.choices(vocab, weights=probs, k=1)[0]
print(next_word)

A SyntaxError because random.choices is used incorrectly

B "hello" always because it is first in vocab

C "world" most likely, but could be any word from vocab

D "machine" always because it has the highest probability

💡 Hint

random.choices picks based on weights but can pick any word.

The code uses random.choices with weights to pick the next word. "world" has the highest chance but other words can still be picked randomly.

❓ Model Choice

advanced

Choosing a model type for creative text generation

You want to build a system that writes creative stories with varied vocabulary and style. Which model type is best?

A A simple Markov chain model based on word transitions

B A bag-of-words model that ignores word order

C A rule-based system with fixed sentence templates

D A large transformer-based language model trained on diverse text

💡 Hint

Consider which model can capture complex language patterns and creativity.

Transformer models learn deep language patterns and context, enabling creative and varied text generation. Markov chains and rule-based systems are limited and less flexible.

❓ Metrics

advanced

Evaluating text generation quality

Which metric best measures how well a text generation model produces fluent and coherent sentences?

A BLEU score comparing generated text to reference text

B Mean squared error between predicted and actual word embeddings

C Accuracy of classifying text sentiment

D Number of words generated per second

💡 Hint

Think about metrics that compare generated text to human-written examples.

BLEU score measures similarity between generated and reference texts, reflecting fluency and coherence. Other options do not evaluate text quality directly.

🔧 Debug

expert

Why does this text generation code produce repetitive output?

Consider this code snippet generating text word by word. It produces repetitive phrases like "the the the...". What is the most likely cause?

NLP

import random

vocab = ['the', 'cat', 'sat', 'on', 'mat']
probs = [0.9, 0.025, 0.025, 0.025, 0.025]

output = []
for _ in range(5):
    next_word = random.choices(vocab, weights=probs, k=1)[0]
    output.append(next_word)
print(' '.join(output))

A The probability distribution is too skewed, causing the model to pick "the" repeatedly.

B The vocab list is too small to generate varied text.

C random.choices is not suitable for text generation and causes repetition.

D The loop runs only 5 times, so repetition is expected.

💡 Hint

Look at the probabilities assigned to each word.

The high probability (0.9) for "the" makes it very likely to be chosen repeatedly, causing repetitive output. Balancing probabilities helps generate varied text.