Temperature and sampling in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Effect of Temperature on Sampling Distribution

In natural language generation, the temperature parameter controls randomness in sampling. What happens to the output distribution when the temperature is set very high (e.g., 10)?

A The distribution becomes nearly uniform, making all words almost equally likely.

B The distribution becomes very peaked, favoring the most probable word strongly.

C The sampling ignores the model probabilities and picks words randomly from the vocabulary.

D The model always picks the word with the highest probability deterministically.

💡 Hint

Think about how raising temperature affects the sharpness of probabilities.

High temperature flattens the probability distribution, making all options closer to equal chance, increasing randomness.

❓ Predict Output

intermediate

Output of Sampling with Temperature

Given the logits array and temperature, what is the output probability distribution after applying softmax with temperature?

NLP

import numpy as np

def softmax_with_temperature(logits, temperature):
    scaled_logits = logits / temperature
    exp_logits = np.exp(scaled_logits - np.max(scaled_logits))
    return exp_logits / exp_logits.sum()

logits = np.array([2.0, 1.0, 0.1])
temperature = 1.0
probs = softmax_with_temperature(logits, temperature)
print(probs.round(3))

A [0.576 0.298 0.126]

B [0.500 0.300 0.200]

C [0.843 0.128 0.029]

D [0.659 0.242 0.099]

💡 Hint

Lower temperature sharpens the distribution, increasing the highest probability.

Lower temperature sharpens the distribution, increasing the highest probability. The output matches option D.

❓ Model Choice

advanced

Choosing Sampling Strategy for Creative Text Generation

You want to generate creative and diverse text using a language model. Which sampling strategy and temperature setting is best?

A Beam search with beam width 10 and temperature 0.5

B Top-k sampling with k=5 and temperature 1.0

C Random sampling with temperature 5.0

D Greedy sampling with temperature 0.1

💡 Hint

Consider how top-k limits choices and temperature controls randomness.

Top-k sampling with moderate temperature balances creativity and coherence by sampling from the top probable words with some randomness.

❓ Hyperparameter

advanced

Impact of Temperature on Model Output Diversity

During training a text generation model, you observe that increasing temperature from 0.7 to 1.5 changes output diversity. What is the expected effect?

A Output quality improves with fewer grammatical errors.

B Output becomes more deterministic and repetitive.

C Output becomes more random and diverse, with less repetition.

D Output length drastically decreases.

💡 Hint

Think about how temperature affects randomness in sampling.

Higher temperature increases randomness, leading to more diverse and less repetitive outputs.

❓ Metrics

expert

Evaluating Sampling Temperature Effects on Perplexity

You generate text samples from a language model at different temperatures and compute perplexity on a fixed test set. Which temperature setting is likely to yield the lowest perplexity?

A Temperature 0.1

B Temperature 1.0

C Temperature 2.0

D Temperature 5.0

💡 Hint

Lower perplexity means the model predicts test data better; consider how temperature affects prediction confidence.

Lower temperature sharpens the distribution, making predictions closer to the most likely tokens, reducing perplexity.