FastText embeddings in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

How does FastText handle out-of-vocabulary words?

FastText can create embeddings for words not seen during training. How does it do this?

A It uses a dictionary of synonyms to find a known word vector.

B It ignores unknown words and assigns them a zero vector.

C It breaks words into character n-grams and sums their vectors to form the word vector.

D It randomly initializes a vector for unknown words each time.

💡 Hint

Think about how FastText uses parts of words to build vectors.

FastText represents words as a sum of character n-gram vectors, allowing it to generate embeddings for unseen words by combining known subword units.

❓ Predict Output

intermediate

Output of FastText vector dimension

What is the output of the following code snippet?

NLP

from gensim.models import FastText
sentences = [['hello', 'world'], ['machine', 'learning']]
model = FastText(sentences, vector_size=10, window=3, min_count=1, epochs=5)
print(len(model.wv['hello']))

A 5

B 10

C 3

D 1

💡 Hint

Check the vector_size parameter used when creating the model.

The vector_size parameter sets the dimension of word vectors. Here it is 10, so the length of the vector for 'hello' is 10.

❓ Model Choice

advanced

Choosing FastText for morphologically rich languages

You want to build word embeddings for a language with many word forms and suffixes. Which model is best suited?

A Word2Vec, because it learns embeddings only from whole words.

B One-hot encoding, because it is simple and effective.

C GloVe, because it uses global co-occurrence statistics.

D FastText, because it uses subword information to handle word variations.

💡 Hint

Consider how subword information helps with many word forms.

FastText uses character n-grams, which helps it capture morphological variations common in rich languages, unlike Word2Vec or GloVe.

❓ Metrics

advanced

Evaluating FastText embeddings with analogy tasks

You evaluate FastText embeddings on a word analogy task (e.g., king - man + woman = ?). Which metric best measures performance?

A Accuracy of correctly predicted analogy words.

B Mean squared error between vectors.

C Perplexity of the embedding model.

D Loss value during training.

💡 Hint

Think about how analogy tasks are scored.

Analogy tasks are evaluated by accuracy: how often the model predicts the correct word completing the analogy.

🔧 Debug

expert

Debugging FastText training convergence issue

You train a FastText model but notice the loss does not decrease after many epochs. Which is the most likely cause?

A The learning rate is too high, causing the model to overshoot minima.

B The vector size is too large, causing overfitting.

C The window size is too small, so context is ignored.

D The training data is too large, causing slow convergence.

💡 Hint

Consider how learning rate affects training stability.

A too high learning rate can cause the model to jump around the loss surface without settling, preventing loss decrease.