GloVe embeddings in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What does GloVe embeddings capture?

GloVe embeddings are used to represent words as vectors. What key information do these vectors capture?

A The global co-occurrence statistics of words in a corpus

B The frequency of each word in the corpus only

C Only the syntactic structure of sentences

D Random numeric values assigned to words

💡 Hint

Think about how GloVe uses word pairs appearing together in the whole text.

GloVe embeddings are trained by analyzing how often words appear together across the entire text corpus, capturing semantic relationships.

❓ Predict Output

intermediate

Output of GloVe vector lookup

Given a loaded GloVe embedding dictionary glove_vectors where keys are words and values are vectors, what is the output of the code below?

NLP

word = 'king'
vector = glove_vectors.get(word)
print(len(vector))

A 100

B 300

C 50

D KeyError

💡 Hint

Common GloVe embeddings come in sizes like 50, 100, 200, or 300 dimensions.

The standard GloVe embeddings often have 300 dimensions, so the vector length for 'king' is 300.

❓ Model Choice

advanced

Choosing GloVe for a sentiment analysis task

You want to build a sentiment analysis model on movie reviews. Which reason best justifies choosing pre-trained GloVe embeddings over training embeddings from scratch?

A GloVe embeddings only capture syntax, which is irrelevant for sentiment.

B GloVe embeddings are always smaller in size, making the model faster regardless of data.

C GloVe embeddings are trained on large corpora and capture rich semantic relations, improving model performance with less data.

D Training embeddings from scratch always leads to better results than using GloVe.

💡 Hint

Think about the benefits of using embeddings trained on large text collections.

Pre-trained GloVe embeddings have learned word meanings from huge datasets, so they help models understand words better, especially when your own data is limited.

❓ Hyperparameter

advanced

Effect of embedding dimension size in GloVe

When training GloVe embeddings, what is the effect of increasing the embedding dimension size from 50 to 300?

A It decreases the model's ability to learn word meanings.

B It makes the embeddings sparse and less useful.

C It has no effect on the quality or size of embeddings.

D It increases the ability to capture complex word relationships but requires more memory and computation.

💡 Hint

Think about trade-offs between vector size and information captured.

Larger embedding dimensions allow capturing more detailed word relationships but increase memory use and computation time.

❓ Metrics

expert

Evaluating GloVe embeddings quality

You trained GloVe embeddings on a custom corpus. Which metric best helps evaluate if the embeddings capture meaningful semantic relationships?

A Cosine similarity between vectors of related word pairs

B Mean squared error of embedding vectors

C Number of unique words in the corpus

D Training loss of a separate classification model

💡 Hint

Think about how to measure if similar words have similar vectors.

Cosine similarity measures how close vectors are in direction, so high similarity for related words shows embeddings capture meaning well.

Practice

1. What is the main purpose of GloVe embeddings in natural language processing?

easy

A. To generate random text based on input

B. To translate text from one language to another

C. To count the frequency of words in a document

D. To convert words into numerical vectors that capture meaning and relationships

Solution

1. Step 1: Understand what embeddings do

   Embeddings convert words into numbers so machines can understand text.

2. Step 2: Identify GloVe's role

   GloVe embeddings specifically capture word meanings and relationships in vector form.

3. Final Answer:

   To convert words into numerical vectors that capture meaning and relationships -> Option D

4. Quick Check:

   GloVe = word vectors capturing meaning [OK]

Hint: Remember: embeddings = words to numbers showing meaning [OK]

Common Mistakes:

• Confusing embeddings with translation
• Thinking embeddings count word frequency
• Assuming embeddings generate text

2. Which of the following is the correct way to load pre-trained GloVe embeddings in Python using the gensim library?

easy

A. glove = gensim.models.FastText.load('glove.txt')

B. glove = gensim.models.Word2Vec.load('glove.txt')

C. glove = gensim.models.KeyedVectors.load_word2vec_format('glove.txt', binary=False)

D. glove = gensim.load('glove.txt')

Solution

1. Step 1: Recall GloVe loading method

   GloVe embeddings are loaded as KeyedVectors using load_word2vec_format with binary=False.

2. Step 2: Check options for correct syntax

   glove = gensim.models.KeyedVectors.load_word2vec_format('glove.txt', binary=False) uses the correct function and parameters for GloVe format.

3. Final Answer:

   glove = gensim.models.KeyedVectors.load_word2vec_format('glove.txt', binary=False) -> Option C

4. Quick Check:

   Use load_word2vec_format with binary=False for GloVe [OK]

Hint: Use load_word2vec_format with binary=False for GloVe files [OK]

Common Mistakes:

• Using Word2Vec.load for GloVe files
• Forgetting binary=False parameter
• Using FastText load for GloVe

3. Given the following Python code snippet using pre-trained GloVe embeddings, what will be the output?

from gensim.models import KeyedVectors

glove = KeyedVectors.load_word2vec_format('glove.6B.50d.txt', binary=False)
result = glove.similarity('king', 'queen')
print(round(result, 2))

medium

A. 0.00

B. 0.78

C. 1.00

D. -0.50

Solution

1. Step 1: Understand similarity method

   The similarity method returns a cosine similarity score between two word vectors, usually between 0 and 1 for related words.

2. Step 2: Interpret expected similarity for 'king' and 'queen'

   These words are closely related, so the similarity is high but less than 1, typically around 0.78.

3. Final Answer:

   0.78 -> Option B

4. Quick Check:

   Similarity('king','queen') ≈ 0.78 [OK]

Hint: Related words have similarity close to but less than 1 [OK]

Common Mistakes:

• Assuming similarity is always 1 for related words
• Confusing similarity with distance
• Expecting negative similarity for related words

4. You try to find the vector for the word 'unseenword' using GloVe embeddings with this code:

vector = glove['unseenword']

But it raises a KeyError. What is the best way to fix this error?

medium

A. Check if the word exists in the embeddings before accessing it

B. Use glove.get_vector('unseenword') without checking

C. Ignore the error and continue

D. Restart the Python kernel

Solution

1. Step 1: Understand cause of KeyError

   The word 'unseenword' is not in the GloVe vocabulary, so direct access raises KeyError.

2. Step 2: Use safe access method

   Check if the word exists using 'if word in glove' before accessing to avoid errors.

3. Final Answer:

   Check if the word exists in the embeddings before accessing it -> Option A

4. Quick Check:

   Check word presence before access to avoid KeyError [OK]

Hint: Always check word in embeddings before access [OK]

Common Mistakes:

• Trying to access vectors without checking existence
• Ignoring errors instead of handling them
• Restarting kernel does not fix missing words

5. You want to improve a text classification model by using GloVe embeddings. Which approach best combines GloVe vectors with your model to handle words not in the GloVe vocabulary?

hard

A. Initialize an embedding layer with GloVe vectors and allow it to be trainable with random vectors for unknown words

B. Use only GloVe vectors and ignore unknown words during training

C. Replace unknown words with a fixed zero vector and freeze the embedding layer

D. Train a new embedding from scratch without using GloVe

Solution

1. Step 1: Understand embedding layer initialization

   Initializing with GloVe vectors provides good starting word representations.

2. Step 2: Handle unknown words and training

   Allowing the embedding layer to be trainable lets the model learn vectors for unknown words starting from random initialization.

3. Final Answer:

   Initialize an embedding layer with GloVe vectors and allow it to be trainable with random vectors for unknown words -> Option A

4. Quick Check:

   Trainable embeddings + GloVe + random unknown vectors = best practice [OK]

Hint: Use trainable embeddings with GloVe plus random unknown vectors [OK]

Common Mistakes:

• Ignoring unknown words instead of learning their vectors
• Freezing embeddings and losing adaptability
• Not using pre-trained GloVe vectors at all