Temperature and sampling in NLP - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to apply temperature scaling to logits before sampling.

NLP

scaled_logits = logits / [1]

Drag options to blanks, or click blank then click option'

Asoftmax

Blogits

Ctemperature

Dsampling

Attempts:

3 left

💡 Hint

Common Mistakes

Using logits directly without scaling.

Multiplying logits by temperature instead of dividing.

Temperature is used to scale logits before sampling to control randomness.

2fill in blank

medium

Complete the code to sample an index from the probability distribution after softmax.

NLP

probabilities = softmax(scaled_logits)
sampled_index = [1](len(probabilities), p=probabilities)

Drag options to blanks, or click blank then click option'

Anp.random.choice

Bargmax

Crandom.choice

Dmax

Attempts:

3 left

💡 Hint

Common Mistakes

Using argmax which always picks the highest probability.

Using max which returns the maximum value, not an index.

np.random.choice can sample an index based on given probabilities.

3fill in blank

hard

Fix the error in the code to correctly apply temperature and sample from logits.

NLP

def sample_with_temperature(logits, temperature):
    scaled = logits * [1]
    probs = softmax(scaled)
    return np.random.choice(len(probs), p=probs)

Drag options to blanks, or click blank then click option'

Alogits

B1/temperature

Ctemperature

Dnp.exp

Attempts:

3 left

💡 Hint

Common Mistakes

Multiplying logits directly by temperature instead of dividing.

Not scaling logits before softmax.

Logits must be divided by temperature, which is equivalent to multiplying by 1/temperature.

4fill in blank

hard

Fill both blanks to create a function that returns probabilities after temperature scaling.

NLP

def temperature_scaled_probs(logits, [1]):
    scaled_logits = logits / [2]
    return softmax(scaled_logits)

Drag options to blanks, or click blank then click option'

Atemperature

Blogits

Dprobabilities

Attempts:

3 left

💡 Hint

Common Mistakes

Using logits as parameter instead of temperature.

Not dividing logits by temperature.

The function takes temperature as input and divides logits by it before softmax.

5fill in blank

hard

Fill all three blanks to implement sampling with temperature scaling and softmax.

NLP

def sample(logits, [1]):
    scaled = logits / [2]
    probs = softmax([3])
    return np.random.choice(len(probs), p=probs)

Drag options to blanks, or click blank then click option'

Atemperature

Cscaled

Dlogits

Attempts:

3 left

💡 Hint

Common Mistakes

Using logits instead of scaled logits in softmax.

Not dividing logits by temperature.

The function uses temperature to scale logits, applies softmax to scaled logits, then samples.