Bird
0
0

Which Python code snippet correctly computes attention weights using the dot product of query Q and keys K before applying softmax?

easy📝 Syntax Q3 of 15
NLP - Sequence Models for NLP
Which Python code snippet correctly computes attention weights using the dot product of query Q and keys K before applying softmax?
Ascores = np.dot(K, Q) weights = np.exp(scores) * np.sum(np.exp(scores))
Bscores = np.dot(Q.T, K) weights = np.sum(np.exp(scores))
Cscores = np.dot(Q, K.T) weights = np.exp(scores) / np.sum(np.exp(scores))
Dscores = np.dot(Q, K) weights = np.exp(scores) / np.exp(np.sum(scores))
Step-by-Step Solution
Solution:

  1. Step 1: Compute dot product

    Attention scores are computed as Q multiplied by K transposed.

  2. Step 2: Apply softmax

    Softmax normalizes exponentiated scores by dividing by their sum.

  3. Final Answer:

    scores = np.dot(Q, K.T)\nweights = np.exp(scores) / np.sum(np.exp(scores)) -> Option C

  4. Quick Check:

    Dot product with K.T then softmax [OK]
Quick Trick: Dot Q with K.T then softmax normalize [OK]
Common Mistakes:
MISTAKES
  • Not transposing K before dot product
  • Incorrect normalization in softmax
  • Using multiplication instead of division for softmax

Want More Practice?

15+ quiz questions · All difficulty levels · Free

Free Signup - Practice All Questions
More NLP Quizzes
Sentiment Analysis Advanced - Aspect-based sentiment analysis - Quiz 5medium Sequence Models for NLP - Embedding layer usage - Quiz 13medium Sequence Models for NLP - GRU for text - Quiz 2easy Sequence Models for NLP - Padding and sequence length - Quiz 7medium Text Similarity and Search - Semantic similarity with embeddings - Quiz 8hard Text Similarity and Search - Semantic similarity with embeddings - Quiz 11easy Topic Modeling - Choosing number of topics - Quiz 10hard Topic Modeling - LDA with scikit-learn - Quiz 8hard Word Embeddings - Pre-trained embedding usage - Quiz 11easy Word Embeddings - GloVe embeddings - Quiz 13medium