N-grams in NLP

Introduction

N-grams help us understand how words appear together in text. They show sequences of words to find patterns or predict the next word.

To predict the next word when typing on a phone keyboard.

To find common phrases in customer reviews.

To improve search engines by understanding word pairs or triples.

To detect spam messages by spotting unusual word combinations.

To analyze writing style by looking at frequent word sequences.

Syntax

NLP

from sklearn.feature_extraction.text import CountVectorizer

vectorizer = CountVectorizer(ngram_range=(n, n))
X = vectorizer.fit_transform(corpus)
print(vectorizer.get_feature_names_out())

ngram_range=(n, n) means you get only n-grams of size n (like bigrams if n=2).

You can set ngram_range=(1, 2) to get both single words and pairs.

Examples

This gets single words (unigrams) from the sentence.

NLP

from sklearn.feature_extraction.text import CountVectorizer

corpus = ['I love machine learning']
vectorizer = CountVectorizer(ngram_range=(1, 1))
X = vectorizer.fit_transform(corpus)
print(vectorizer.get_feature_names_out())

This gets pairs of words (bigrams) from the sentence.

NLP

from sklearn.feature_extraction.text import CountVectorizer

corpus = ['I love machine learning']
vectorizer = CountVectorizer(ngram_range=(2, 2))
X = vectorizer.fit_transform(corpus)
print(vectorizer.get_feature_names_out())

This gets both single words and pairs of words.

NLP

from sklearn.feature_extraction.text import CountVectorizer

corpus = ['I love machine learning']
vectorizer = CountVectorizer(ngram_range=(1, 2))
X = vectorizer.fit_transform(corpus)
print(vectorizer.get_feature_names_out())

Sample Model

This code finds all pairs of words (bigrams) in the three sentences. It prints the list of bigrams and how many times each appears in each sentence.

NLP

from sklearn.feature_extraction.text import CountVectorizer

corpus = [
    'I love machine learning',
    'Machine learning is fun',
    'I love coding in Python'
]

# Create bigrams only
vectorizer = CountVectorizer(ngram_range=(2, 2))
X = vectorizer.fit_transform(corpus)

# Show the bigrams found
bigrams = vectorizer.get_feature_names_out()
print('Bigrams:', bigrams)

# Show the count matrix as array
counts = X.toarray()
print('Counts matrix:\n', counts)

Important Notes

N-grams help capture context by looking at word groups, not just single words.

Higher n (like 3 or 4) means longer word sequences but fewer matches and more data needed.

CountVectorizer automatically lowercases words and removes punctuation by default.

Summary

N-grams are groups of n words appearing together in text.

They help find patterns and improve text predictions.

Use CountVectorizer with ngram_range to extract n-grams easily.

Practice

1. What is an n-gram in natural language processing?

easy

A. A random selection of n words from a text

B. A single word repeated n times

C. A sentence with n words

D. A group of n consecutive words in a text

Solution

1. Step 1: Understand the definition of n-gram

   An n-gram is defined as a sequence of n consecutive words appearing together in text.

2. Step 2: Compare options with definition

   Only A group of n consecutive words in a text correctly describes an n-gram as consecutive words, not random or repeated words.

3. Final Answer:

   A group of n consecutive words in a text -> Option D

4. Quick Check:

   n-gram = consecutive words [OK]

Hint: Remember: n-gram means consecutive words, not random ones [OK]

Common Mistakes:

• Thinking n-gram means repeated words
• Confusing n-gram with sentence length
• Assuming words are randomly picked

2. Which of the following is the correct way to set up a CountVectorizer to extract bigrams in Python?

easy

A. CountVectorizer(ngram_range=(1,1))

B. CountVectorizer(ngram_range=(2,2))

C. CountVectorizer(ngram_range=(0,2))

D. CountVectorizer(ngram_range=(1,3))

Solution

1. Step 1: Understand ngram_range parameter

   ngram_range=(2,2) extracts only bigrams (groups of exactly 2 words).

2. Step 2: Evaluate each option

   CountVectorizer(ngram_range=(1,1)) extracts unigrams only; C is invalid because 0 is not a valid n; D extracts unigrams to trigrams.

3. Final Answer:

   CountVectorizer(ngram_range=(2,2)) -> Option B

4. Quick Check:

   bigrams = ngram_range (2,2) [OK]

Hint: Set ngram_range=(2,2) for only bigrams [OK]

Common Mistakes:

• Using (1,1) which extracts unigrams
• Using (0,2) which is invalid
• Using (1,3) which extracts multiple n-grams

3. What will be the output tokens when extracting trigrams from the sentence 'I love machine learning' using CountVectorizer(ngram_range=(3,3))?

medium

A. ['I love machine', 'love machine learning']

B. ['I love', 'love machine', 'machine learning']

C. ['I', 'love', 'machine', 'learning']

D. ['I love machine learning']

Solution

1. Step 1: Understand trigram extraction

   Trigrams are groups of 3 consecutive words. The sentence has 4 words, so possible trigrams are words 1-3 and 2-4.

2. Step 2: List trigrams from the sentence

   First trigram: 'I love machine', second trigram: 'love machine learning'.

3. Final Answer:

   ['I love machine', 'love machine learning'] -> Option A

4. Quick Check:

   Trigrams = groups of 3 words [OK]

Hint: Count groups of 3 consecutive words for trigrams [OK]

Common Mistakes:

• Listing bigrams instead of trigrams
• Listing single words instead of groups
• Combining all words as one token

4. Identify the error in this code snippet for extracting bigrams:

from sklearn.feature_extraction.text import CountVectorizer
text = ['hello world']
vectorizer = CountVectorizer(ngram_range=(1,2))
vectorizer.fit_transform(text)
print(vectorizer.get_feature_names())

medium

A. The text should be a string, not a list

B. The ngram_range should be (2,2) to extract only bigrams

C. The method get_feature_names() is deprecated and should be get_feature_names_out()

D. CountVectorizer cannot extract bigrams

Solution

1. Step 1: Check method usage

   In recent sklearn versions, get_feature_names() is deprecated; get_feature_names_out() is the correct method.

2. Step 2: Validate other parts

   ngram_range=(1,2) is valid for unigrams and bigrams; text as list is correct; CountVectorizer supports bigrams.

3. Final Answer:

   get_feature_names() is deprecated and should be get_feature_names_out() -> Option C

4. Quick Check:

   Use get_feature_names_out() for features [OK]

Hint: Use get_feature_names_out() instead of deprecated get_feature_names() [OK]

Common Mistakes:

• Thinking ngram_range=(1,2) is wrong for bigrams
• Assuming text must be a string, not list
• Believing CountVectorizer can't extract bigrams

5. You want to build a text prediction model that uses both unigrams and bigrams but excludes any n-grams containing stop words like 'the' or 'and'. Which approach is best?

hard

A. Use CountVectorizer with ngram_range=(1,2) and stop_words='english'

B. Use CountVectorizer with ngram_range=(2,2) and no stop words removal

C. Use CountVectorizer with ngram_range=(1,1) and manually remove stop words after extraction

D. Use CountVectorizer with ngram_range=(1,3) and stop_words=None

Solution

1. Step 1: Understand requirements

   We need unigrams and bigrams, and want to exclude stop words in any n-gram.

2. Step 2: Evaluate options

   Use CountVectorizer with ngram_range=(1,2) and stop_words='english' uses ngram_range=(1,2) for unigrams and bigrams and removes stop words automatically. Others either miss unigrams, include stop words, or include trigrams.

3. Final Answer:

   Use CountVectorizer with ngram_range=(1,2) and stop_words='english' -> Option A

4. Quick Check:

   Unigrams + bigrams + stop word removal = Use CountVectorizer with ngram_range=(1,2) and stop_words='english' [OK]

Hint: Set ngram_range and stop_words='english' to filter stop words [OK]

Common Mistakes:

• Not removing stop words from bigrams
• Using wrong ngram_range missing unigrams
• Including trigrams when not needed