N-gram language models in NLP - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is an N-gram in language modeling?

An N-gram is a sequence of N words used together to predict the next word in a sentence. For example, a bigram is two words, a trigram is three words.

beginner

How does an N-gram language model predict the next word?

It looks at the previous N-1 words and calculates the probability of each possible next word based on how often those words appeared together in training data.

beginner

What is the difference between unigram, bigram, and trigram models?

Unigram models consider single words independently. Bigram models consider pairs of words. Trigram models consider triples of words to predict the next word.

intermediate

Why do N-gram models face the problem of data sparsity?

Because many word combinations may not appear in the training data, making it hard to estimate their probabilities accurately.

intermediate

What is smoothing in N-gram language models?

Smoothing is a technique to adjust probabilities so that unseen word sequences get a small, non-zero probability instead of zero, helping the model handle new phrases.

What does a trigram model use to predict the next word?

A The previous two words

B The previous three words

C Only the last word

D All words in the sentence

A trigram model uses the previous two words (N-1) to predict the next word.

Why is smoothing important in N-gram models?

A To remove rare words from the model

B To assign zero probability to unseen word sequences

C To increase the size of the training data

D To assign small probabilities to unseen word sequences

Smoothing assigns small non-zero probabilities to unseen sequences, preventing zero probability issues.

Which problem occurs because many word sequences are rare or missing in training data?

A Overfitting

B Data sparsity

C Underfitting

D Bias

Data sparsity happens when many word combinations are rare or missing, making probability estimation hard.

In a bigram model, what is the probability of a word based on?

A Random chance

B The word after it

C The word before it

D The entire sentence

A bigram model predicts a word based on the immediately preceding word.

Which of these is NOT a type of N-gram model?

A Quadrigram

B Unigram

C Trigram

D Bigram

The correct term is 'four-gram' or '4-gram', not 'quadrigram'.

Explain how an N-gram language model predicts the next word in a sentence.

Think about how you guess the next word when typing a message.

Describe the challenges N-gram models face and how smoothing helps.

Consider what happens when the model sees a new phrase it never learned.