Temperature and sampling parameters in Prompt Engineering / GenAI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Effect of Temperature on Model Output Diversity

When using a language model with a temperature parameter, what happens to the model's output as the temperature increases from 0.1 to 1.0?

A The output becomes more random and diverse, allowing more creative responses.

B The output becomes more deterministic and repetitive, reducing creativity.

C The model stops generating any output because the temperature is too high.

D The output length increases automatically with higher temperature.

💡 Hint

Think about how temperature controls randomness in choosing words.

Higher temperature values increase randomness in the model's word choices, making outputs more diverse and creative. Lower temperatures make the model pick the most likely words, producing more predictable text.

❓ Predict Output

intermediate

Sampling with Top-k Parameter

Given a language model sampling with top-k=3, which option best describes the effect on word selection?

A The model samples from all words with probability above 0.3.

B The model always picks the single most probable word.

C The model only considers the top 3 most probable next words for sampling.

D The model ignores the top 3 words and samples from the rest.

💡 Hint

Top-k limits the candidate words to a fixed number.

Top-k sampling restricts the model to choose the next word only from the top k most likely words, increasing output diversity while avoiding very unlikely words.

❓ Hyperparameter

advanced

Choosing Temperature for Balanced Creativity

You want your language model to generate responses that are creative but still coherent. Which temperature value is most suitable?

A 3.0

B 0.5

C 1.5

D 0.01

💡 Hint

Very low temperatures make output boring; very high make it chaotic.

A temperature around 0.5 balances randomness and coherence, producing creative yet sensible outputs. Very low values produce repetitive text; very high values produce nonsensical text.

❓ Metrics

advanced

Impact of Sampling Parameters on Perplexity

How does increasing temperature during sampling affect the perplexity metric measured on generated text?

A Perplexity generally increases because outputs become less predictable.

B Perplexity decreases because outputs become more accurate.

C Perplexity remains unchanged as it depends only on training data.

D Perplexity becomes zero because the model guesses perfectly.

💡 Hint

Higher randomness means less predictable text.

Higher temperature increases randomness, making generated text less predictable and thus increasing perplexity, which measures uncertainty in predictions.

🔧 Debug

expert

Unexpected Output with Temperature and Top-p Sampling

Consider this code snippet using a language model with temperature=0.8 and top-p=0.9. The output is unexpectedly repetitive and lacks diversity. Which is the most likely cause?

Prompt Engineering / GenAI

model.generate(prompt, temperature=0.8, top_p=0.9, top_k=0)

A The model is ignoring temperature when top-p is set.

B Temperature 0.8 is too low to allow any randomness.

C Top-k is set to 0, which disables top-k filtering, causing less diversity.

D Top-p 0.9 is too restrictive, limiting word choices too much.

💡 Hint

Top-p controls cumulative probability cutoff for candidate words.

Top-p=0.9 means sampling only from the smallest set of words whose probabilities sum to 90%. If this set is very small, it limits diversity despite temperature. Top-k=0 disables top-k filtering, so it does not reduce diversity.

Practice

1. What does the temperature parameter control in AI text generation?

easy

A. The speed of the AI's response

B. The length of the generated text

C. How random or focused the AI's answers are

D. The number of words the AI can use

Solution

1. Step 1: Understand the role of temperature

   The temperature parameter adjusts randomness in AI output. A low temperature makes answers more focused and predictable, while a high temperature increases randomness and creativity.

2. Step 2: Match the description to the options

   Only How random or focused the AI's answers are correctly describes temperature as controlling randomness or focus in AI answers.

3. Final Answer:

   How random or focused the AI's answers are -> Option C

4. Quick Check:

   Temperature controls randomness = A [OK]

Hint: Temperature means randomness level in AI output [OK]

Common Mistakes:

• Confusing temperature with output length
• Thinking temperature controls speed
• Mixing temperature with vocabulary size

2. Which of the following is the correct way to set the temperature to 0.7 in a Python API call for text generation?

easy

A. generate_text(temperature=0.7)

B. generate_text(temp=7)

C. generate_text(temperature='0.7')

D. generate_text(temperature=7)

Solution

1. Step 1: Identify correct parameter name and type

   The parameter controlling randomness is named temperature and expects a float value between 0 and 1 (commonly).

2. Step 2: Check each option

   generate_text(temperature=0.7) uses the correct parameter name and a valid float value 0.7. generate_text(temp=7) uses wrong parameter name and integer 7. generate_text(temperature='0.7') passes a string instead of float. generate_text(temperature=7) uses an invalid integer 7 instead of a float between 0 and 1.

3. Final Answer:

   generate_text(temperature=0.7) -> Option A

4. Quick Check:

   Correct parameter and float value = D [OK]

Hint: Use parameter name 'temperature' with float value [OK]

Common Mistakes:

• Using wrong parameter name like 'temp'
• Passing temperature as string instead of float
• Using values outside 0-1 range

3. Given this code snippet:

response = generate_text(prompt='Hello', temperature=0.1, top_p=0.9)
print(response)

What is the expected behavior of the AI output?

medium

A. Highly random and creative text

B. Very focused and predictable text

C. Text with many rare words

D. Text ignoring the prompt

Solution

1. Step 1: Analyze temperature value

   A temperature of 0.1 is very low, so the AI output will be focused and predictable, avoiding randomness.

2. Step 2: Analyze top_p value

   Top-p of 0.9 means the AI considers the most probable words covering 90% probability, further limiting randomness.

3. Final Answer:

   Very focused and predictable text -> Option B

4. Quick Check:

   Low temperature + top_p = focused output [OK]

Hint: Low temperature means focused output, not random [OK]

Common Mistakes:

• Assuming low temperature means more creativity
• Ignoring top_p effect on word choice
• Thinking AI ignores prompt with low temperature

4. You set temperature=1.5 in your AI call but get an error. What is the likely cause and fix?

medium

A. Temperature must be an integer; set it to 1

B. Temperature must be a string; set it to '1.5'

C. Temperature cannot be set; remove it

D. Temperature must be between 0 and 1; set it to 0.9

Solution

1. Step 1: Understand valid temperature range

   Temperature values are usually required to be between 0 and 1 to control randomness properly.

2. Step 2: Identify error cause and fix

   Setting temperature to 1.5 is outside the valid range, causing an error. Fix is to set it to a valid value like 0.9.

3. Final Answer:

   Temperature must be between 0 and 1; set it to 0.9 -> Option D

4. Quick Check:

   Temperature range 0-1 = B [OK]

Hint: Temperature must be 0 to 1, not above [OK]

Common Mistakes:

• Using values above 1 for temperature
• Thinking temperature must be integer
• Passing temperature as string

5. You want the AI to generate creative but coherent stories. Which combination of temperature and top_p is best?

hard

A. temperature=0.8, top_p=0.9

B. temperature=0.2, top_p=0.5

C. temperature=1.0, top_p=0.1

D. temperature=0.0, top_p=1.0

Solution

1. Step 1: Understand desired output style

   Creative but coherent stories need some randomness (creativity) but also focus (coherence).

2. Step 2: Evaluate each parameter combination

   temperature=0.8, top_p=0.9 has temperature 0.8 (moderately high randomness) and top_p 0.9 (limits to likely words), balancing creativity and coherence. temperature=0.2, top_p=0.5 is too low randomness, temperature=1.0, top_p=0.1 has very low top_p causing incoherence, temperature=0.0, top_p=1.0 is zero randomness, so very dull.

3. Final Answer:

   temperature=0.8, top_p=0.9 -> Option A

4. Quick Check:

   Moderate temperature + high top_p = creative & coherent [OK]

Hint: Use moderate temperature and high top_p for creative coherence [OK]

Common Mistakes:

• Choosing zero temperature for creativity
• Using very low top_p causing odd word choices
• Setting temperature too high causing nonsense