FastText embeddings in NLP - Cheat Sheet & Quick Revision

Recall & Review

beginner

What are FastText embeddings?

FastText embeddings are word representations that include information about subwords (small parts of words). This helps capture meanings of rare or new words by looking at their parts.

intermediate

How does FastText differ from traditional word embeddings like Word2Vec?

Unlike Word2Vec, which treats each word as a single unit, FastText breaks words into smaller pieces called n-grams. This helps it understand words it has never seen before by combining these pieces.

beginner

Why are subword embeddings useful in FastText?

Subword embeddings help FastText handle rare words, misspellings, and new words by learning from smaller parts of words. This makes the model more flexible and accurate.

intermediate

What is the role of n-grams in FastText embeddings?

N-grams are sequences of characters inside words. FastText learns embeddings for these n-grams and combines them to form the word's embedding, capturing internal word structure.

advanced

How can FastText embeddings improve performance on languages with rich morphology?

Because FastText uses subword information, it can better understand word variations and endings common in languages with rich morphology, improving representation and downstream task results.

What is the main advantage of FastText embeddings over Word2Vec?

A They ignore word order

B They require less training data

C They only work with fixed vocabulary

D They use subword information to handle rare words

FastText embeddings use subword (n-gram) information, allowing them to represent rare or unseen words better than Word2Vec.

In FastText, what are n-grams?

A Complete sentences

B Sequences of characters inside words

C Individual words

D Paragraphs

N-grams in FastText are sequences of characters within words, used to build subword embeddings.

Why does FastText perform well on misspelled words?

A Because it uses a dictionary lookup

B Because it ignores spelling

C Because it uses subword parts to build embeddings

D Because it trains on misspelled words only

FastText uses subword embeddings, so even if a word is misspelled, its parts can still be recognized and represented.

Which of these is NOT a feature of FastText embeddings?

A Ignoring word context completely

B Handling out-of-vocabulary words

C Using character n-grams

D Capturing subword information

FastText embeddings do consider word context during training; they do not ignore it.

FastText embeddings are especially useful for which type of languages?

A Languages with rich morphology

B Languages with no grammar

C Languages with only short words

D Languages without alphabets

FastText's use of subword information helps it handle languages with many word forms and endings, known as rich morphology.

Explain how FastText embeddings use subword information to represent words.

Think about how smaller parts of words help understand the whole word.

Describe why FastText embeddings can improve performance on languages with many word forms.

Consider how word parts change in different forms.