Find the bug in this code snippet for attention output: import numpy as np Q = np.array([1, 0]) K = np.array([[1, 0], [0, 1]]) V = np.array([[5, 5], [10, 10]]) scores = np.dot(Q, K.T) weights = np.exp(scores) weights /= np.sum(weights) output = np.dot(weights, V.T) print(output)

medium📝 Debug Q7 of 15

NLP - Sequence Models for NLP

AScores calculation is wrong

BValues matrix should not be transposed in final dot product

CSoftmax calculation is incorrect

DWeights are not normalized

Step-by-Step Solution

Solution:

Step 1: Review final dot product
Weights shape is (2,), V shape is (2,2). Dot(weights, V.T) results in shape (2,), but values should be weighted by weights directly.
Step 2: Correct final dot product
Use np.dot(weights, V) without transpose to get correct output vector.
Final Answer:
Values matrix should not be transposed in final dot product -> Option B
Quick Check:
Dot weights with values, no transpose [OK]

Quick Trick: Do not transpose values when multiplying by weights [OK]

Common Mistakes:

MISTAKES

Transposing values wrongly
Incorrect softmax
Wrong score calculation

Master "Sequence Models for NLP" in NLP

9 interactive learning modes - each teaches the same concept differently

Learn Why Deep Model Try Challenge Experiment Recall Metrics

Want More Practice?

15+ quiz questions · All difficulty levels · Free

Free Signup - Practice All Questions

More NLP Quizzes

Find the bug in this code snippet for attention output: import numpy as np Q = np.array([1, 0]) K = np.array([[1, 0], [0, 1]]) V = np.array([[5, 5], [10, 10]]) scores = np.dot(Q, K.T) weights = np.exp(scores) weights /= np.sum(weights) output = np.dot(weights, V.T) print(output)

Step 1: Review final dot product

Step 2: Correct final dot product

Final Answer:

Quick Check:

Want More Practice?