Word similarity and analogies in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding cosine similarity in word embeddings

Which statement best describes what cosine similarity measures between two word vectors?

A The difference in magnitude between the two vectors, ignoring their direction

B The Euclidean distance between the two vectors, showing how far apart they are in space

C The sum of the element-wise multiplication of the two vectors without normalization

D The angle between the two vectors, indicating how similar their directions are regardless of length

💡 Hint

Think about how cosine similarity focuses on direction rather than length.

Cosine similarity measures the cosine of the angle between two vectors. It shows how similar their directions are, ignoring their lengths. This is useful in word embeddings because words with similar meanings tend to point in similar directions.

❓ Predict Output

intermediate

Output of word analogy vector operation

Given the following word vectors, what is the result of the operation: vector('king') - vector('man') + vector('woman')? Vectors: king = [0.8, 0.6] man = [0.7, 0.5] woman = [0.6, 0.7]

NLP

import numpy as np

king = np.array([0.8, 0.6])
man = np.array([0.7, 0.5])
woman = np.array([0.6, 0.7])

result = king - man + woman
print(result)

A [0.7 0.8]

B [0.9 0.8]

C [0.6 0.7]

D [0.7 0.6]

💡 Hint

Perform the vector addition and subtraction step by step.

Subtracting 'man' from 'king' and then adding 'woman' results in [0.8-0.7+0.6, 0.6-0.5+0.7] = [0.7, 0.8].

❓ Model Choice

advanced

Choosing the best model for capturing word analogies

Which word embedding model is best known for capturing linear relationships that allow solving analogies like 'king' - 'man' + 'woman' = 'queen'?

A Bag-of-Words model

B Word2Vec Skip-gram model

C TF-IDF vectorizer

D One-hot encoding

💡 Hint

Think about models that learn dense vector representations with semantic relationships.

Word2Vec Skip-gram learns dense vectors where semantic relationships are encoded as linear offsets, enabling analogy tasks. Bag-of-Words and TF-IDF do not capture such relationships, and one-hot encoding is sparse and does not encode meaning.

❓ Metrics

advanced

Evaluating word similarity with Spearman correlation

You have a list of human-rated word similarity scores and cosine similarity scores from your model. Which metric best measures how well your model's similarity rankings match human judgments?

A Spearman rank correlation coefficient

B Mean squared error

C Accuracy

D Precision

💡 Hint

Consider metrics that compare rankings rather than exact values.

Spearman rank correlation measures how well the order of similarity scores from the model matches the order of human judgments, which is ideal for similarity tasks. Mean squared error measures exact value differences, which is less meaningful here.

🔧 Debug

expert

Debugging incorrect analogy results

You run this code to find the word closest to the vector result of 'king' - 'man' + 'woman' but get an unrelated word. What is the most likely cause? Code snippet: import numpy as np result = embeddings['king'] - embeddings['man'] + embeddings['woman'] closest_word = None max_sim = -1 for word, vec in embeddings.items(): sim = np.dot(result, vec) / (np.linalg.norm(result) * np.linalg.norm(vec)) if sim > max_sim: max_sim = sim closest_word = word print(closest_word)

A The embeddings dictionary includes the words 'king', 'man', and 'woman', so no issue there

B The embeddings vectors are not normalized before the operation, causing invalid results

C The code does not exclude the input words from the search, so it may return 'king' or 'woman' instead of the correct analogy

D The cosine similarity calculation is incorrect because it uses dot product without normalization

💡 Hint

Think about what happens if the search includes the original words used in the vector operation.

The code searches over all words including 'king', 'man', and 'woman'. Since the result vector is close to these, it may return one of them instead of the intended analogy word. Excluding input words from the search fixes this.

Practice

1. What does word similarity measure in natural language processing?

easy

A. How close two words are in meaning using numbers

B. How often two words appear together in a sentence

C. The length difference between two words

D. The number of letters two words share

Solution

1. Step 1: Understand the concept of word similarity

   Word similarity measures how close two words are in meaning, often represented by a number like cosine similarity.

2. Step 2: Differentiate from other word properties

   Frequency or letter count does not capture meaning closeness, so those options are incorrect.

3. Final Answer:

   How close two words are in meaning using numbers -> Option A

4. Quick Check:

   Word similarity = meaning closeness [OK]

Hint: Similarity means meaning closeness, not letter or frequency count [OK]

Common Mistakes:

• Confusing similarity with word frequency
• Thinking similarity is about word length
• Assuming similarity counts shared letters

2. Which of the following is the correct way to find the cosine similarity between two word vectors vec1 and vec2 in Python using NumPy?

easy

A. np.dot(vec1, vec2) / (np.linalg.norm(vec1) * np.linalg.norm(vec2))

B. np.dot(vec1, vec2) * (np.linalg.norm(vec1) + np.linalg.norm(vec2))

C. np.dot(vec1, vec2) - (np.linalg.norm(vec1) * np.linalg.norm(vec2))

D. np.dot(vec1, vec2) / (np.linalg.norm(vec1) + np.linalg.norm(vec2))

Solution

1. Step 1: Recall cosine similarity formula

   Cosine similarity = dot product of vectors divided by product of their norms.

2. Step 2: Match formula to code

   np.dot(vec1, vec2) / (np.linalg.norm(vec1) * np.linalg.norm(vec2)) matches the formula exactly using np.dot and np.linalg.norm.

3. Final Answer:

   np.dot(vec1, vec2) / (np.linalg.norm(vec1) * np.linalg.norm(vec2)) -> Option A

4. Quick Check:

   Cosine similarity = dot / (norm1 * norm2) [OK]

Hint: Cosine similarity divides dot product by product of norms [OK]

Common Mistakes:

• Adding norms instead of multiplying
• Subtracting norms from dot product
• Multiplying dot product by sum of norms

3. Given the following word vectors:

king = [0.5, 0.8, 0.3]
queen = [0.45, 0.75, 0.35]
man = [0.6, 0.7, 0.2]
woman = [0.55, 0.65, 0.25]

What is the closest word to the vector king - man + woman?

medium

A. king

B. man

C. queen

D. woman

Solution

1. Step 1: Calculate the vector for king - man + woman

   Subtract man from king: [0.5-0.6, 0.8-0.7, 0.3-0.2] = [-0.1, 0.1, 0.1]. Add woman: [-0.1+0.55, 0.1+0.65, 0.1+0.25] = [0.45, 0.75, 0.35].

2. Step 2: Compare result to known vectors

   The resulting vector matches queen exactly: [0.45, 0.75, 0.35].

3. Final Answer:

   queen -> Option C

4. Quick Check:

   king - man + woman = queen [OK]

Hint: king - man + woman equals queen vector [OK]

Common Mistakes:

• Not subtracting man vector before adding woman
• Mixing up vector addition order
• Choosing original words instead of analogy result

4. The following code tries to find the word most similar to king - man + woman but has a flaw:

import numpy as np
words = {'king': np.array([0.5, 0.8, 0.3]), 'queen': np.array([0.45, 0.75, 0.35]), 'man': np.array([0.6, 0.7, 0.2]), 'woman': np.array([0.55, 0.65, 0.25])}
result = words['king'] - words['man'] + words['woman']
max_word = None
max_sim = -1
for word, vec in words.items():
    sim = np.dot(result, vec) / (np.linalg.norm(result) * np.linalg.norm(vec))
    if sim > max_sim:
        max_word = word
print(max_word)

What is the main flaw?

medium

A. The variable max_sim is initialized incorrectly

B. Division by zero occurs due to zero vector norm

C. The dot product is computed without normalizing vectors

D. The code does not exclude the original words from similarity search

Solution

1. Step 1: Analyze the similarity search loop

   The loop compares the result vector to all words including 'king', 'man', and 'woman' which are part of the calculation.

2. Step 2: Understand why this is problematic

   Including original words can cause the highest similarity to be the input words themselves, which is usually unwanted and can cause misleading results.

3. Final Answer:

   The code does not exclude the original words from similarity search -> Option D

4. Quick Check:

   Exclude input words to avoid bias [OK]

Hint: Exclude input words from similarity search to avoid bias [OK]

Common Mistakes:

• Assuming zero division error without checking norms
• Thinking max_sim initialization causes error
• Ignoring normalization in dot product

5. You want to find the word that fits the analogy: Paris is to France as Tokyo is to ? Using pre-trained word vectors, which approach is best to find the answer?

hard

A. Calculate vector: France - Tokyo + Paris, then find closest word vector

B. Calculate vector: Tokyo - Paris + France, then find closest word vector

C. Calculate vector: Paris + France - Tokyo, then find closest word vector

D. Calculate vector: Tokyo + Paris - France, then find closest word vector

Solution

1. Step 1: Understand analogy vector arithmetic

   Analogies use the formula: word2 - word1 + word3 to find the missing word. Here, Paris is word1, France is word2, Tokyo is word3.

2. Step 2: Apply formula to this analogy

   Calculate Tokyo - Paris + France to get the vector representing the answer.

3. Final Answer:

   Calculate vector: Tokyo - Paris + France, then find closest word vector -> Option B

4. Quick Check:

   Analogy vector = word3 - word1 + word2 [OK]

Hint: Use analogy formula: word3 - word1 + word2 [OK]

Common Mistakes:

• Swapping order of subtraction and addition
• Adding all vectors without subtraction
• Using wrong words in formula