N-gram language models in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding the purpose of N-gram models

What is the main purpose of using an N-gram language model in natural language processing?

A To generate images from text descriptions using neural networks

B To translate text from one language to another using deep learning

C To cluster documents based on their topics using unsupervised learning

D To predict the next word in a sequence based on the previous N-1 words

💡 Hint

Think about what N-gram models estimate about word sequences.

N-gram models estimate the probability of a word given the previous N-1 words, helping predict the next word in a sequence.

❓ Predict Output

intermediate

Output of a bigram probability calculation

Given the sentence: 'I love machine learning', and the bigram counts: {('I', 'love'): 3, ('love', 'machine'): 2, ('machine', 'learning'): 4}, what is the bigram probability P('machine'|'love') using maximum likelihood estimation?

NLP

bigram_counts = {('I', 'love'): 3, ('love', 'machine'): 2, ('machine', 'learning'): 4}
unigram_counts = {'I': 3, 'love': 4, 'machine': 4}
prob = bigram_counts[('love', 'machine')] / unigram_counts['love']
print(prob)

A 0.5

B 0.33

C 0.67

D 1.0

💡 Hint

Divide the count of the bigram by the count of the first word in the bigram.

The bigram probability P('machine'|'love') = count('love machine') / count('love') = 2 / 4 = 0.5

❓ Hyperparameter

advanced

Choosing the value of N in N-gram models

Which of the following is a common trade-off when increasing the value of N in an N-gram language model?

A Higher N always improves model accuracy without any drawbacks

B Higher N leads to better context capture but requires more data and increases sparsity

C Higher N reduces model size and speeds up training

D Higher N eliminates the need for smoothing techniques

💡 Hint

Think about how longer sequences affect data requirements and sparsity.

Increasing N captures more context but causes data sparsity and needs more training data, making option B correct.

❓ Metrics

advanced

Evaluating N-gram language models with perplexity

If an N-gram language model has a perplexity of 50 on a test set, what does this indicate about the model's performance?

A The model is uncertain and predicts the test data poorly

B The model has zero error on the test data

C The model is very confident and predicts the test data well

D The model's predictions are random and unrelated to the test data

💡 Hint

Lower perplexity means better prediction; higher means worse.

Perplexity measures how well a model predicts data; higher perplexity means more uncertainty and worse performance.

🔧 Debug

expert

Identifying the error in smoothing implementation

Consider this code snippet for add-one smoothing in a bigram model: ```python bigram_counts = {('the', 'cat'): 3, ('cat', 'sat'): 2} unigram_counts = {'the': 5, 'cat': 4} vocab_size = 10 word1 = 'cat' word2 = 'sat' prob = (bigram_counts.get((word1, word2), 0) + 1) / (unigram_counts[word1] + vocab_size) print(prob) ``` What error will this code raise when calculating the probability for the bigram ('sat', 'on')?

A ZeroDivisionError because unigram_counts[word1] is zero

B TypeError because bigram_counts.get returns None

C KeyError because 'sat' is not in unigram_counts

D No error, code runs correctly

💡 Hint

Check if all words used as keys exist in unigram_counts.

If 'sat' is not in unigram_counts, accessing unigram_counts[word1] raises KeyError.