Text chunking strategies in Prompt Engineering / GenAI - Model Pipeline Trace

Model Pipeline - Text chunking strategies

This pipeline breaks long text into smaller, manageable pieces called chunks. These chunks help AI models understand and process text better by focusing on smaller parts at a time.

Data Flow - 5 Stages

1Input Text

1 document x 1000 words→Receive raw text document→1 document x 1000 words

"Once upon a time in a faraway land, there was a small village surrounded by mountains..."

↓

2Preprocessing

1 document x 1000 words→Clean text (remove punctuation, lowercase)→1 document x 980 words

"once upon a time in a faraway land there was a small village surrounded by mountains"

↓

3Chunking

1 document x 980 words→Split text into chunks of 100 words with 20 words overlap→12 chunks x 100 words each

Chunk 1: words 1-100, Chunk 2: words 81-180, ..., Chunk 12: words 881-980

↓

4Feature Extraction

12 chunks x 100 words→Convert each chunk into numerical vectors (embeddings)→12 chunks x 100-dimensional vectors

[0.12, 0.45, ..., 0.33] for chunk 1

↓

5Model Input

12 chunks x 100-dimensional vectors→Feed chunks into AI model for understanding or generation→Model processes each chunk independently

Model predicts sentiment or answers questions per chunk

Training Trace - Epoch by Epoch

Loss
1.0 |****
0.8 |*** 
0.6 |**  
0.4 |*   
0.2 |    
0.0 +----
     1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.85	0.60	Model starts learning from chunked text with moderate accuracy
2	0.65	0.72	Loss decreases and accuracy improves as model adapts to chunked inputs
3	0.50	0.80	Model shows good understanding of chunks, accuracy rising
4	0.40	0.85	Training converges with lower loss and higher accuracy
5	0.35	0.88	Final epoch shows stable performance on chunked text

Prediction Trace - 4 Layers

Layer 1: Input Chunk

Layer 2: Embedding Layer

Layer 3: Model Processing

Layer 4: Output Aggregation

Model Quiz - 3 Questions

Why do we use overlapping words between chunks?

A To keep context between chunks

B To reduce the total number of chunks

C To make chunks shorter

D To remove stop words

Key Insight

Breaking long text into overlapping chunks helps AI models keep context and understand text better. Training shows that chunking improves learning by making text easier to process in parts.