Top-p and top-k sampling in Prompt Engineering / GenAI - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to set the top-k value for sampling.

Prompt Engineering / GenAI

top_k = [1]

Drag options to blanks, or click blank then click option'

A0.9

B50

C1000

DNone

Attempts:

3 left

💡 Hint

Common Mistakes

Using a float value like 0.9 for top-k instead of an integer.

Setting top-k to None disables it, not sets a value.

Top-k sampling uses an integer value to limit the number of tokens considered. Here, 50 means only the top 50 tokens are sampled from.

2fill in blank

medium

Complete the code to set the top-p value for nucleus sampling.

Prompt Engineering / GenAI

top_p = [1]

Drag options to blanks, or click blank then click option'

A50

B100

C0.9

DNone

Attempts:

3 left

💡 Hint

Common Mistakes

Using an integer like 50 for top-p instead of a float.

Setting top-p to None disables it, not sets a value.

Top-p sampling uses a probability threshold between 0 and 1. Here, 0.9 means sampling from tokens whose cumulative probability is at least 90%.

3fill in blank

hard

Fix the error in the code to apply top-k sampling correctly.

Prompt Engineering / GenAI

output = model.generate(input_ids, do_sample=True, top_k=[1])

Drag options to blanks, or click blank then click option'

ATrue

BNone

C0.9

D50

Attempts:

3 left

💡 Hint

Common Mistakes

Passing a float like 0.9 causes an error.

Passing None disables top-k but does not apply sampling.

Top-k must be an integer specifying how many top tokens to sample from. 50 is a valid integer value.

4fill in blank

hard

Fill both blanks to apply top-p and top-k sampling together.

Prompt Engineering / GenAI

output = model.generate(input_ids, do_sample=True, top_k=[1], top_p=[2])

Drag options to blanks, or click blank then click option'

A50

B0.9

C0.8

D100

Attempts:

3 left

💡 Hint

Common Mistakes

Swapping top-k and top-p values.

Using floats for top-k or integers for top-p.

Top-k is set to 50 tokens, and top-p is set to 0.9 probability threshold. Both control sampling diversity.

5fill in blank

hard

Fill all three blanks to create a dictionary of sampling parameters with top-k, top-p, and temperature.

Prompt Engineering / GenAI

sampling_params = {'top_k': [1], 'top_p': [2], 'temperature': [3]

Drag options to blanks, or click blank then click option'

A40

B0.85

C1.0

D0.7

Attempts:

3 left

💡 Hint

Common Mistakes

Using temperature as an integer.

Mixing up top-p and temperature values.

Top-k is 40 tokens, top-p is 0.85 probability, and temperature is 0.7 to control randomness.

Practice

1. What does top-k sampling do in text generation?

easy

A. It selects the next word from the top k most likely words.

B. It selects the next word randomly from all possible words.

C. It picks words until their total probability reaches p.

D. It always picks the single most likely next word.

Solution

1. Step 1: Understand top-k sampling definition

   Top-k sampling limits choices to the top k words with highest probabilities.

2. Step 2: Compare with other methods

   Random selection from all possible words and picking words until total probability reaches p describe other methods; always picking the single most likely next word is greedy decoding, not sampling.

3. Final Answer:

   It selects the next word from the top k most likely words. -> Option A

4. Quick Check:

   Top-k = top k words [OK]

Hint: Top-k means pick from top k words only [OK]

Common Mistakes:

• Confusing top-k with top-p sampling
• Thinking top-k picks only one word always
• Mixing top-k with greedy decoding

2. Which of the following is the correct way to apply top-p sampling in code?

easy

A. Select words until their cumulative probability exceeds p.

B. Select exactly p words with highest probabilities.

C. Select the single word with probability p.

D. Select words randomly ignoring probabilities.

Solution

1. Step 1: Recall top-p sampling definition

   Top-p sampling chooses the smallest set of words whose total probability is at least p.

2. Step 2: Evaluate options

   Selecting words until their cumulative probability exceeds p matches this definition. Selecting exactly p words confuses top-p with top-k. Random selection ignoring probabilities and selecting a single word with probability p are incorrect.

3. Final Answer:

   Select words until their cumulative probability exceeds p. -> Option A

4. Quick Check:

   Top-p = cumulative probability ≥ p [OK]

Hint: Top-p sums probabilities to reach p [OK]

Common Mistakes:

• Confusing number of words with cumulative probability
• Thinking top-p picks fixed number of words
• Ignoring word probabilities in selection

3. Given these word probabilities sorted descending: {'a': 0.4, 'b': 0.3, 'c': 0.2, 'd': 0.1}, what words are included in top-p sampling with p=0.7?

medium

A. ['a']

B. ['a', 'b', 'c']

C. ['a', 'b']

D. ['a', 'b', 'c', 'd']

Solution

1. Step 1: Calculate cumulative probabilities

   Sum probabilities in order: 'a' = 0.4, 'a'+'b' = 0.7, 'a'+'b'+'c' = 0.9.

2. Step 2: Select smallest set ≥ p=0.7

   The smallest set with sum ≥ 0.7 is ['a', 'b'].

3. Final Answer:

   ['a', 'b'] -> Option C

4. Quick Check:

   Cumulative sum ≥ 0.7 includes 'a' and 'b' [OK]

Hint: Sum probabilities until ≥ p [OK]

Common Mistakes:

• Including too many words beyond p
• Stopping before reaching p
• Confusing top-p with top-k count

4. You wrote code for top-k sampling but it always picks only one word. What is the likely bug?

medium

A. You summed probabilities instead of sorting words.

B. You set k=1 instead of a larger number.

C. You used top-p sampling code instead of top-k.

D. You forgot to normalize probabilities.

Solution

1. Step 1: Understand top-k parameter effect

   Setting k=1 means only the single most likely word is chosen.

2. Step 2: Check other options

   Summing probabilities or mixing methods won't cause always one word; normalization affects probabilities but not count.

3. Final Answer:

   You set k=1 instead of a larger number. -> Option B

4. Quick Check:

   k=1 picks only one word [OK]

Hint: Check if k=1 limits output to one word [OK]

Common Mistakes:

• Confusing top-k and top-p parameters
• Ignoring parameter values in code
• Assuming normalization fixes count

5. You want to generate text that balances creativity and coherence. Which approach is best?

hard

A. Use random sampling ignoring probabilities.

B. Use greedy decoding to always pick the most likely word.

C. Use top-k sampling with k=1 only.

D. Use top-k sampling with a moderate k and top-p sampling with p around 0.9 together.

Solution

1. Step 1: Understand creativity vs coherence tradeoff

   Greedy decoding is too rigid; random sampling is too chaotic; top-k with k=1 is greedy.

2. Step 2: Combine top-k and top-p for balance

   Using moderate k and p near 0.9 limits choices to plausible words but allows variety, improving naturalness.

3. Final Answer:

   Use top-k sampling with a moderate k and top-p sampling with p around 0.9 together. -> Option D

4. Quick Check:

   Combining top-k and top-p balances randomness and coherence [OK]

Hint: Combine top-k and top-p for best text quality [OK]

Common Mistakes:

• Choosing greedy decoding for creativity
• Ignoring probability thresholds
• Using too small k or p values