N-grams in NLP - Model Pipeline Trace

Model Pipeline - N-grams

This pipeline shows how text data is transformed into N-grams, which are groups of consecutive words. These N-grams help the model understand word patterns to make predictions or analyze text.

Data Flow - 5 Stages

1Raw Text Input

1000 sentences x variable length→Collect raw sentences as input data→1000 sentences x variable length

"I love machine learning"

↓

2Text Cleaning

1000 sentences x variable length→Lowercase and remove punctuation→1000 sentences x variable length

"i love machine learning"

↓

3Tokenization

1000 sentences x variable length→Split sentences into words (tokens)→1000 sentences x variable length tokens

["i", "love", "machine", "learning"]

↓

4N-gram Generation (n=2)

1000 sentences x variable length tokens→Create pairs of consecutive words (bigrams)→1000 sentences x (variable length - 1) bigrams

["i love", "love machine", "machine learning"]

↓

5Vectorization

1000 sentences x variable length bigrams→Convert bigrams into numerical features (counts)→1000 rows x 5000 bigram features

Feature vector with counts of each bigram

Training Trace - Epoch by Epoch

Loss
1.0 |          *
0.8 |        *  
0.6 |      *    
0.4 |    *      
0.2 |  *        
0.0 +-----------
     1 2 3 4 5  Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.85	0.6	Model starts learning word patterns from bigrams
2	0.65	0.72	Loss decreases and accuracy improves as model learns
3	0.5	0.8	Model captures important bigram features
4	0.4	0.85	Training converges with good accuracy
5	0.35	0.88	Final epoch shows stable improvement

Prediction Trace - 6 Layers

Layer 1: Input Sentence

Layer 2: Text Cleaning

Layer 3: Tokenization

Layer 4: N-gram Generation (bigrams)

Layer 5: Vectorization

Layer 6: Model Prediction

Model Quiz - 3 Questions

What does the N in N-grams represent?

A Number of consecutive words grouped together

B Number of sentences processed

C Number of characters in a word

D Number of punctuation marks removed

Key Insight

N-grams help models understand word order and context by grouping words together. Converting these groups into numbers allows the model to learn patterns that improve predictions over time.