Word embeddings concept (Word2Vec) in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Word2Vec Training Objective

Which of the following best describes the training objective of the Word2Vec Skip-gram model?

A Predict the center word given the context words.

B Cluster words into groups based on their frequency.

C Reduce the dimensionality of word vectors using PCA.

D Predict the context words given the center word.

💡 Hint

Think about what the model tries to guess during training.

The Skip-gram model learns to predict surrounding context words given a center word, helping it capture word relationships.

❓ Predict Output

intermediate

Output of Word2Vec Vector Similarity Calculation

Given the following Python code using Gensim's Word2Vec, what is the output of the similarity calculation?

ML Python

from gensim.models import Word2Vec
sentences = [['cat', 'sat', 'on', 'the', 'mat'], ['dog', 'barked', 'at', 'the', 'cat']]
model = Word2Vec(sentences, vector_size=10, window=2, min_count=1, epochs=10)
sim = model.wv.similarity('cat', 'dog')
print(round(sim, 2))

A A float value close to 0.5

B A float value close to 1.0

C A float value close to -1.0

D Raises a KeyError because 'dog' is not in vocabulary

💡 Hint

Both 'cat' and 'dog' appear in the sentences, so similarity is computed.

The similarity between 'cat' and 'dog' is positive but less than 1 because they appear in related contexts but are different words.

❓ Model Choice

advanced

Choosing the Right Word2Vec Architecture

You want to train word embeddings that capture rare word relationships well in a small dataset. Which Word2Vec architecture is best?

A Hierarchical clustering model

B Skip-gram model

C Principal Component Analysis (PCA) on one-hot vectors

D Continuous Bag of Words (CBOW) model

💡 Hint

One architecture is better at learning from rare words.

The Skip-gram model performs better on rare words because it predicts context from a single word, allowing it to learn better representations for infrequent words.

❓ Hyperparameter

advanced

Effect of Window Size in Word2Vec

What is the effect of increasing the window size parameter in Word2Vec training?

A The model considers a larger context around each word, capturing broader semantic relationships.

B The model trains faster because it uses fewer context words.

C The model ignores stop words during training.

D The model reduces the vector size to speed up training.

💡 Hint

Window size controls how many words around the center word are used.

A larger window size means the model looks at more surrounding words, helping it learn broader meanings but possibly less syntactic detail.

❓ Metrics

expert

Evaluating Word2Vec Embeddings Quality

Which metric is commonly used to evaluate the quality of trained Word2Vec embeddings on a word analogy task?

A Silhouette score of word clusters

B Cross-entropy loss on training data

C Accuracy of correctly answered analogy questions

D Mean Squared Error between embeddings

💡 Hint

Think about how well embeddings solve analogy puzzles like 'king - man + woman = ?'

Accuracy on analogy tasks measures how often the embeddings correctly solve word relationship puzzles, reflecting semantic quality.

Practice

1. What is the main purpose of Word2Vec in machine learning?

easy

A. To translate text from one language to another

B. To count the number of words in a sentence

C. To convert words into numerical vectors that capture their meanings

D. To generate random words for testing

Solution

1. Step 1: Understand Word2Vec's role

   Word2Vec creates vectors representing words based on their context, capturing meaning.

2. Step 2: Compare options

   Only To convert words into numerical vectors that capture their meanings describes converting words into meaningful numerical vectors.

3. Final Answer:

   To convert words into numerical vectors that capture their meanings -> Option C

4. Quick Check:

   Word2Vec = word vectors [OK]

Hint: Word2Vec turns words into numbers showing meaning [OK]

Common Mistakes:

• Thinking Word2Vec translates languages
• Confusing Word2Vec with word counting
• Assuming Word2Vec generates random words

2. Which of the following is the correct way to import Word2Vec from the gensim library in Python?

easy

A. import gensim.Word2Vec

B. import Word2Vec from gensim.models

C. from gensim import Word2Vec

D. from gensim.models import Word2Vec

Solution

1. Step 1: Recall correct Python import syntax

   Python uses 'from module import class' to import specific classes.

2. Step 2: Match with gensim syntax

   The correct import is 'from gensim.models import Word2Vec'. Other options have wrong syntax.

3. Final Answer:

   from gensim.models import Word2Vec -> Option D

4. Quick Check:

   Correct import syntax = from gensim.models import Word2Vec [OK]

Hint: Use 'from module import class' for specific imports [OK]

Common Mistakes:

• Using 'import Word2Vec from ...' which is invalid syntax
• Trying to import directly from 'gensim' without 'models'
• Using dot notation in import statements incorrectly

3. Given the following code snippet, what will be the output of print(model.wv.most_similar('king', topn=1))?

from gensim.models import Word2Vec
sentences = [['king', 'queen', 'man', 'woman'], ['prince', 'princess', 'boy', 'girl']]
model = Word2Vec(sentences, vector_size=10, window=2, min_count=1, epochs=10)

medium

A. [('man', similarity_score)] where similarity_score is close to 0

B. [('queen', similarity_score)] where similarity_score is close to 1

C. Error because 'king' is not in vocabulary

D. Empty list []

Solution

1. Step 1: Check if 'king' is in vocabulary

   'king' appears in the sentences, so it is in the model's vocabulary.

2. Step 2: Understand most_similar output

   Word2Vec finds words with similar context. 'queen' is closely related to 'king' in the data, so it will have highest similarity.

3. Final Answer:

   [('queen', similarity_score)] where similarity_score is close to 1 -> Option B

4. Quick Check:

   Most similar to 'king' = 'queen' [OK]

Hint: Most similar words share context, 'queen' pairs with 'king' [OK]

Common Mistakes:

• Assuming 'king' is missing from vocabulary
• Expecting unrelated words like 'man' as most similar
• Thinking output is an error or empty list

4. Identify the error in this Word2Vec training code and select the fix:

from gensim.models import Word2Vec
sentences = [['cat', 'dog', 'mouse'], ['dog', 'cat', 'elephant']]
model = Word2Vec(sentences, vector_size=5, window=2, min_count=2, epochs=5)
model.train(sentences, total_examples=1, epochs=5)

medium

A. Set min_count=1 to include all words in training

B. Change total_examples to len(sentences) in train()

C. Remove the train() call because Word2Vec trains on initialization

D. Increase vector_size to at least 10

Solution

1. Step 1: Analyze min_count effect

   min_count=2 excludes words appearing less than twice. 'mouse' and 'elephant' appear once, so they are ignored.

2. Step 2: Fix by lowering min_count

   Setting min_count=1 includes all words, fixing vocabulary size and training coverage.

3. Final Answer:

   Set min_count=1 to include all words in training -> Option A

4. Quick Check:

   min_count controls word inclusion [OK]

Hint: min_count filters words; lower it to include rare words [OK]

Common Mistakes:

• Ignoring min_count effect on vocabulary
• Thinking train() is unnecessary or wrongly used
• Changing vector_size without addressing vocabulary issue

5. You want to find the word that best fits the analogy: 'king' is to 'man' as 'queen' is to ____. Using Word2Vec, which operation would you perform to find this word?

hard

A. vector('king') - vector('man') + vector('queen')

B. vector('king') + vector('man') - vector('queen')

C. vector('queen') - vector('king') + vector('man')

D. vector('man') - vector('king') + vector('queen')

Solution

1. Step 1: Understand analogy vector math

   Word2Vec analogies use vector arithmetic: 'king' - 'man' + 'queen' approximates the vector for the missing word.

2. Step 2: Apply formula to find the answer

   Performing vector('king') - vector('man') + vector('queen') finds the word similar to 'queen' but related like 'king' is to 'man'.

3. Final Answer:

   vector('king') - vector('man') + vector('queen') -> Option A

4. Quick Check:

   Analogy = king - man + queen [OK]

Hint: Use king - man + queen to solve analogies [OK]

Common Mistakes:

• Mixing up vector addition and subtraction order
• Using unrelated vector combinations
• Ignoring analogy vector math concept