Naive Bayes for text in NLP - Model Pipeline Trace

Model Pipeline - Naive Bayes for text

This pipeline shows how a Naive Bayes model learns to classify text messages into categories by counting word frequencies and using probabilities.

Data Flow - 5 Stages

1Raw text data

1000 rows x 1 column→Original text messages with labels→1000 rows x 1 column

"I love this movie" labeled as Positive

↓

2Text cleaning and tokenization

1000 rows x 1 column→Lowercase, remove punctuation, split sentences into words→1000 rows x variable-length word lists

"I love this movie" -> ["i", "love", "this", "movie"]

↓

3Feature extraction (Bag of Words)

1000 rows x variable-length word lists→Count word occurrences, create fixed-size vocabulary vector→1000 rows x 5000 columns

"i love this movie" -> vector with counts for words like 'love':1, 'movie':1

↓

4Train/test split

1000 rows x 5000 columns→Split data into training (80%) and testing (20%) sets→800 rows x 5000 columns (train), 200 rows x 5000 columns (test)

Training set has 800 messages with word count vectors

↓

5Model training (Naive Bayes)

800 rows x 5000 columns→Calculate word probabilities per class with smoothing→Model with learned word probabilities

Probability of word 'love' given Positive class is 0.03

Training Trace - Epoch by Epoch

Loss
0.7 |****
0.6 |****
0.5 |****
0.4 |****
0.3 |****
    +----
     1 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.65	0.7	Initial training with basic word counts
2	0.5	0.8	Model learns better word-class associations
3	0.4	0.85	Improved smoothing and probability estimates
4	0.35	0.88	Model converges with stable accuracy
5	0.33	0.89	Final epoch with slight improvement

Prediction Trace - 4 Layers

Layer 1: Input text

Layer 2: Feature vector creation

Layer 3: Calculate class probabilities

Layer 4: Prediction

Model Quiz - 3 Questions

What does the Bag of Words step do in this pipeline?

A Splits text into sentences

B Counts how many times each word appears in the text

C Removes stop words from the text

D Converts text into audio signals

Key Insight

Naive Bayes uses simple word counts and probabilities to classify text quickly and effectively, showing how counting features can help machines understand language.