Document-term matrix in NLP - Model Pipeline Trace

Model Pipeline - Document-term matrix

A document-term matrix (DTM) is a way to turn text documents into numbers. It shows how often each word appears in each document. This helps computers understand and learn from text.

Data Flow - 5 Stages

1Raw text documents

5 documents x variable length→Collect raw text data→5 documents x variable length

["I love apples", "Apples are tasty", "I eat apples daily", "Tasty apples are good", "Love to eat fruits"]

↓

2Text cleaning

5 documents x variable length→Lowercase, remove punctuation, and extra spaces→5 documents x cleaned text

["i love apples", "apples are tasty", "i eat apples daily", "tasty apples are good", "love to eat fruits"]

↓

3Tokenization

5 documents x cleaned text→Split text into words (tokens)→5 documents x list of tokens

[["i", "love", "apples"], ["apples", "are", "tasty"], ["i", "eat", "apples", "daily"], ["tasty", "apples", "are", "good"], ["love", "to", "eat", "fruits"]]

↓

4Build vocabulary

5 documents x list of tokens→Find unique words across all documents→Vocabulary size: 10 words

["i", "love", "apples", "are", "tasty", "eat", "daily", "good", "to", "fruits"]

↓

5Create document-term matrix

5 documents x list of tokens→Count how many times each word appears in each document→5 documents x 10 words

[[1,1,1,0,0,0,0,0,0,0], [0,0,1,1,1,0,0,0,0,0], [1,0,1,0,0,1,1,0,0,0], [0,0,1,1,1,0,0,1,0,0], [0,1,0,0,0,1,0,0,1,1]]

Training Trace - Epoch by Epoch

Loss
0.9 |*
0.8 |** 
0.7 |***  
0.6 |****  
0.5 |*****   
0.4 |******   
0.3 |*******    
     1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.85	0.50	Initial training with sparse document-term matrix input
2	0.65	0.65	Model learns word patterns better
3	0.50	0.75	Loss decreases and accuracy improves steadily
4	0.40	0.82	Model converging with good performance
5	0.35	0.85	Final epoch shows stable improvement

Prediction Trace - 3 Layers

Layer 1: Input document

Layer 2: Vectorization using document-term matrix

Layer 3: Model prediction

Model Quiz - 3 Questions

What does each row in a document-term matrix represent?

A A document with counts of each word

B A word with counts of each document

C A list of unique words

D A cleaned text sentence

Key Insight

A document-term matrix transforms text into numbers by counting word occurrences. This numeric form allows machine learning models to find patterns in text and improve predictions as training progresses.