What is Temperature in Text Generation in NLP Explained

In NLP text generation, temperature is a setting that controls randomness in the model's output. A low temperature makes the model choose more likely words, producing predictable text, while a high temperature allows more variety and creativity by picking less likely words.

How It Works

Imagine you are choosing words to write a story. If you always pick the most common or expected word, your story might be boring and repetitive. But if you sometimes pick unusual words, your story becomes more creative and surprising. Temperature in text generation works like this choice knob.

Technically, temperature changes how the model turns its confidence scores (called probabilities) into actual word choices. When the temperature is low (close to 0), the model picks the highest probability words almost every time, making the output very predictable. When the temperature is high (above 1), the model spreads out the probabilities more evenly, so it can pick less likely words, adding randomness and creativity.

This helps balance between safe, sensible text and more diverse, interesting text depending on what you want.

Example

This example shows how changing temperature affects the next word prediction probabilities using a simple list of word scores.

python

import numpy as np

def apply_temperature(logits, temperature):
    # Convert logits to probabilities with temperature
    scaled_logits = logits / temperature
    exp_logits = np.exp(scaled_logits - np.max(scaled_logits))
    probs = exp_logits / exp_logits.sum()
    return probs

# Example logits (scores) for 4 possible next words
logits = np.array([2.0, 1.0, 0.1, 0.5])

# Temperatures to test
temperatures = [0.5, 1.0, 2.0]

for temp in temperatures:
    probs = apply_temperature(logits, temp)
    print(f"Temperature: {temp}")
    print(f"Probabilities: {probs}")
    print()

Output

Temperature: 0.5
Probabilities: [0.84379473 0.1141952  0.01010254 0.03190753]

Temperature: 1.0
Probabilities: [0.65900114 0.24243297 0.08904122 0.00952467]

Temperature: 2.0
Probabilities: [0.4042373  0.29301143 0.22422015 0.07853112]

When to Use

Use a low temperature (e.g., 0.2 to 0.7) when you want the text to be clear, focused, and reliable, such as for factual answers or formal writing. This reduces randomness and keeps the output safe.

Use a higher temperature (e.g., 1.0 to 1.5 or more) when you want creative, diverse, or surprising text, like in storytelling, poetry, or brainstorming ideas. This encourages the model to explore less common words and phrases.

Adjusting temperature helps balance between boring but safe text and creative but unpredictable text depending on your goal.

Key Points

• Temperature controls randomness: low means predictable, high means creative.
• It changes word choice probabilities: scaling logits before picking words.
• Use low temperature for accuracy and clarity.
• Use high temperature for creativity and variety.
• Common range is 0.5 to 1.5 depending on task.

Key Takeaways

Temperature controls how random or predictable text generation is in NLP models.

Lower temperature values make the model pick more likely words, producing safer text.

Higher temperature values increase randomness, allowing more creative and diverse outputs.

Adjust temperature based on whether you want clear or creative text.

Typical temperature values range from 0.5 (less random) to 1.5 (more random).