N-gram language models in NLP - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create bigrams from a list of words.

NLP

bigrams = [(words[i], words[i+[1]]) for i in range(len(words)-1)]

Drag options to blanks, or click blank then click option'

A0

B2

C1

D-1

Attempts:

3 left

💡 Hint

Common Mistakes

Using 0 or 2 as the offset causes index errors or wrong pairs.

The bigram pairs each word with the next word, so the index offset is 1.

2fill in blank

medium

Complete the code to count the frequency of each trigram in the text.

NLP

from collections import Counter
trigrams = [(words[i], words[i+1], words[i+[1]]) for i in range(len(words)-2)]
trigram_counts = Counter(trigrams)

Drag options to blanks, or click blank then click option'

A1

B0

C3

D2

Attempts:

3 left

💡 Hint

Common Mistakes

Using 1 or 3 causes incorrect trigram formation or index errors.

For trigrams, the third word is two positions ahead of the first word, so the offset is 2.

3fill in blank

hard

Fix the error in the code that calculates the probability of a bigram using counts.

NLP

bigram_prob = bigram_counts[bigram] / [1]

Drag options to blanks, or click blank then click option'

Asum(bigram_counts.values())

Blen(bigram_counts)

Clen(bigram_counts[bigram])

Dbigram_counts[bigram[0]]

Attempts:

3 left

💡 Hint

Common Mistakes

Using length of bigram_counts or counts of a single word causes wrong probabilities.

The probability of a bigram is its count divided by the total count of all bigrams.

4fill in blank

hard

Fill both blanks to create a dictionary of bigram probabilities from counts.

NLP

bigram_probs = {bg: bigram_counts[bg] / [1] for bg in [2]

Drag options to blanks, or click blank then click option'

Asum(bigram_counts.values())

Bbigram_counts.keys()

Cbigram_counts.items()

Dlen(bigram_counts)

Attempts:

3 left

💡 Hint

Common Mistakes

Using items() instead of keys() causes errors in dictionary comprehension.

We divide each bigram count by the total count of all bigrams (sum of values) and iterate over all bigram keys.

5fill in blank

hard

Fill the two blanks to build a conditional probability dictionary for bigrams.

NLP

cond_probs = {}
for (w1, w2), count in bigram_counts.items():
    w1_total = sum(c for (word1, _), c in bigram_counts.items() if word1 == [1])
    cond_probs.setdefault(w1, {})[[2]] = count / w1_total

Drag options to blanks, or click blank then click option'

Aw2

Bw1

Attempts:

3 left

💡 Hint

Common Mistakes

Mixing up w1 and w2 keys causes wrong dictionary structure or division errors.

The conditional probability P(w2|w1) is count divided by total counts of bigrams starting with w1. We use w2 as the key in the nested dict, divide by w1_total, and sum counts where word1 equals w1.

Practice

1. What does an n-gram language model primarily do?

easy

A. Predict the next word based on previous words

B. Translate text from one language to another

C. Generate images from text descriptions

D. Detect the sentiment of a sentence

Solution

1. Step 1: Understand the purpose of n-gram models

   N-gram models look at sequences of words to predict what comes next.

2. Step 2: Identify the main function

   They use previous words to guess the next word in a sentence.

3. Final Answer:

   Predict the next word based on previous words -> Option A

4. Quick Check:

   N-gram models predict next word = A [OK]

Hint: N-grams predict next word from previous words [OK]

Common Mistakes:

• Confusing n-gram with translation models
• Thinking n-grams generate images
• Mixing up sentiment analysis with n-grams

2. Which of the following is the correct way to represent a bigram from the sentence 'I love AI'?

easy

A. ('AI', 'love')

B. ('I', 'love')

C. ('love', 'AI', 'I')

D. ('I', 'AI')

Solution

1. Step 1: Understand bigrams

   Bigrams are pairs of consecutive words in a sentence.

2. Step 2: Extract bigrams from 'I love AI'

   The pairs are ('I', 'love') and ('love', 'AI'). ('I', 'love') shows a correct bigram.

3. Final Answer:

   ('I', 'love') -> Option B

4. Quick Check:

   Bigram = consecutive word pairs = C [OK]

Hint: Bigrams are pairs of consecutive words [OK]

Common Mistakes:

• Including three words instead of two
• Mixing word order in pairs
• Selecting non-consecutive words

3. Given the sentence 'the cat sat on the mat', what is the count of the trigram ('the', 'cat', 'sat')?

medium

A. 0

B. 2

C. 1

D. 3

Solution

1. Step 1: Identify trigrams in the sentence

   Trigrams are sequences of three consecutive words. The trigrams are: ('the', 'cat', 'sat'), ('cat', 'sat', 'on'), ('sat', 'on', 'the'), ('on', 'the', 'mat').

2. Step 2: Count the trigram ('the', 'cat', 'sat')

   This trigram appears once at the start of the sentence.

3. Final Answer:

   1 -> Option C

4. Quick Check:

   Trigram count = 1 [OK]

Hint: Count exact three-word sequences in order [OK]

Common Mistakes:

• Counting non-consecutive words
• Confusing bigrams with trigrams
• Overcounting repeated words

4. Consider this Python code snippet to generate bigrams from a list of words:

words = ['hello', 'world', 'hello']
bigrams = [(words[i], words[i+1]) for i in range(len(words))]

What error will this code produce?

medium

A. No error, code runs correctly

B. SyntaxError: invalid syntax

C. TypeError: unsupported operand type(s)

D. IndexError: list index out of range

Solution

1. Step 1: Analyze the loop range

   The loop runs from 0 to len(words)-1, which is 0 to 2 for 3 words.

2. Step 2: Check index access inside loop

   At i=2, words[i+1] tries to access words[3], which is out of range, causing IndexError.

3. Final Answer:

   IndexError: list index out of range -> Option D

4. Quick Check:

   Loop index exceeds list length = D [OK]

Hint: Check loop range when accessing i+1 index [OK]

Common Mistakes:

• Using full length in range causing out-of-bounds
• Assuming no error without testing
• Confusing syntax errors with runtime errors

5. You want to build a trigram model from a text corpus but notice many rare trigrams cause sparse data issues. Which technique can help improve your model's predictions?

hard

A. Use smoothing methods like Laplace smoothing

B. Increase the n in n-gram to 5-grams

C. Remove all trigrams that appear less than 10 times

D. Ignore the problem and use raw counts

Solution

1. Step 1: Understand sparse data in n-gram models

   Rare trigrams cause zero or low counts, making predictions unreliable.

2. Step 2: Identify smoothing techniques

   Smoothing like Laplace adds small counts to all n-grams, reducing zero probabilities and improving predictions.

3. Final Answer:

   Use smoothing methods like Laplace smoothing -> Option A

4. Quick Check:

   Smoothing reduces sparse data issues = A [OK]

Hint: Apply smoothing to handle rare n-grams [OK]

Common Mistakes:

• Increasing n worsens sparsity
• Removing rare n-grams loses useful info
• Ignoring sparsity leads to poor predictions