Few-shot learning with prompts in NLP - Model Pipeline Trace

Model Pipeline - Few-shot learning with prompts

This pipeline shows how a language model learns to perform a new task by seeing only a few examples in the prompt. It uses these examples to understand the task and then makes predictions on new inputs.

Data Flow - 4 Stages

1Raw text input

10 samples x 1 text string→Collect few example sentences with labels and a new query sentence→10 samples x 1 text string

"Translate English to French: 'Hello' -> 'Bonjour', 'Goodbye' -> 'Au revoir', Translate English to French: 'Thank you' -> ?"

↓

2Prompt construction

10 samples x 1 text string→Combine few examples and query into a single prompt string→1 sample x 1 long text string

"Translate English to French: 'Hello' -> 'Bonjour', 'Goodbye' -> 'Au revoir', Translate English to French: 'Thank you' ->"

↓

3Model input tokenization

1 sample x 1 long text string→Convert prompt text into tokens for the language model→1 sample x 30 tokens

[Translate, English, to, French, :, 'Hello', ->, 'Bonjour', ',', 'Goodbye', ->, 'Au', 'revoir', ',', Translate, English, to, French, :, 'Thank', 'you', ->]

↓

4Model prediction

1 sample x 30 tokens→Language model predicts next tokens as output→1 sample x 1 token

['Merci']

Training Trace - Epoch by Epoch

Loss
1.2 |****
0.9 |***
0.7 |**
0.5 |*
0.4 |

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.45	Model starts learning from few examples, loss is high, accuracy low
2	0.9	0.60	Loss decreases as model better understands prompt format
3	0.7	0.75	Accuracy improves, model predicts correct translations more often
4	0.5	0.85	Model converges, loss low and accuracy high
5	0.4	0.90	Final epoch shows strong performance on few-shot task

Prediction Trace - 4 Layers

Layer 1: Prompt input

Layer 2: Language model processing

Layer 3: Next token selection

Layer 4: Output generation

Model Quiz - 3 Questions

What does the prompt construction stage do?

A Trains the model on many examples

B Combines few examples and query into one text prompt

C Converts tokens back to text

D Splits text into individual words

Key Insight

Few-shot learning with prompts lets a language model quickly adapt to new tasks by showing just a few examples in the input. The model uses these examples to understand the task format and make accurate predictions without needing full retraining.