Why N-grams in NLP? - Purpose & Use Cases

The Big Idea

What if your computer could guess your next word just by learning common word pairs?

The Scenario

Imagine you want to understand how words appear together in a book to guess the next word someone might say. Doing this by reading every sentence and writing down pairs or triples of words by hand would take forever!

The Problem

Manually tracking word combinations is slow and tiring. It's easy to miss important pairs or triples, and counting them accurately is almost impossible without making mistakes. This makes it hard to analyze language patterns quickly.

The Solution

N-grams automatically break text into groups of words, like pairs or triples, and count how often they appear. This helps computers quickly learn language patterns without any manual counting or guessing.

Before vs After

✗ Before

pairs = {}
words = text.split()
for i in range(len(words)-1):
    pair = (words[i], words[i+1])
    pairs[pair] = pairs.get(pair, 0) + 1

✓ After

from nltk import ngrams
from collections import Counter
pairs = list(ngrams(text.split(), 2))
pair_counts = Counter(pairs)

What It Enables

It lets machines understand and predict language by learning which word groups happen most often.

Real Life Example

When you type a message on your phone, n-grams help predict the next word so your phone can suggest it before you finish typing.

Key Takeaways

Manually tracking word groups is slow and error-prone.

N-grams automatically find and count word groups in text.

This helps machines learn language patterns and make predictions.