Bird
0
0

Which of the following correctly represents the formula to compute attention weights using query (Q) and key (K) vectors?

easy📝 Syntax Q12 of 15
NLP - Sequence Models for NLP
Which of the following correctly represents the formula to compute attention weights using query (Q) and key (K) vectors?
ASigmoid(Q - K)
BSoftmax(Q + K)
CReLU(Q x K)
DSoftmax(Q x K^T)
Step-by-Step Solution
Solution:

  1. Step 1: Recall attention weight calculation

    Attention weights are computed by taking the dot product of query and key vectors, then applying softmax.

  2. Step 2: Match formula to options

    Softmax(Q x K^T) shows softmax applied to Q multiplied by the transpose of K, which is correct.

  3. Final Answer:

    Softmax(Q x K^T) -> Option D

  4. Quick Check:

    Attention weights = softmax(dot product) [OK]
Quick Trick: Attention weights = softmax of query-key dot product [OK]
Common Mistakes:
MISTAKES
  • Adding Q and K instead of dot product
  • Using ReLU or Sigmoid instead of softmax
  • Ignoring transpose on key vector

Want More Practice?

15+ quiz questions · All difficulty levels · Free

Free Signup - Practice All Questions
More NLP Quizzes
Sentiment Analysis Advanced - Aspect-based sentiment analysis - Quiz 5medium Sequence Models for NLP - Embedding layer usage - Quiz 13medium Sequence Models for NLP - GRU for text - Quiz 2easy Sequence Models for NLP - Padding and sequence length - Quiz 7medium Text Similarity and Search - Semantic similarity with embeddings - Quiz 8hard Text Similarity and Search - Semantic similarity with embeddings - Quiz 11easy Topic Modeling - Choosing number of topics - Quiz 10hard Topic Modeling - LDA with scikit-learn - Quiz 8hard Word Embeddings - Pre-trained embedding usage - Quiz 11easy Word Embeddings - GloVe embeddings - Quiz 13medium