Model selection (GPT-4, GPT-3.5) in Prompt Engineering / GenAI - Model Pipeline Trace

Model Pipeline - Model selection (GPT-4, GPT-3.5)

This pipeline compares two language models, GPT-4 and GPT-3.5, to select the best one for a text generation task. It processes input prompts, runs them through both models, evaluates outputs, and chooses the model with better performance.

Data Flow - 5 Stages

1Input prompt

1000 prompts x 1 text column→Receive user text prompts for generation→1000 prompts x 1 text column

"Explain photosynthesis in simple terms."

↓

2Preprocessing

1000 prompts x 1 text column→Tokenize text into model-readable tokens→1000 prompts x variable token length

"Explain photosynthesis" -> [101, 2345, 6789, ...]

↓

3Model inference GPT-3.5

1000 prompts x variable token length→Generate text outputs using GPT-3.5→1000 prompts x generated text

"Photosynthesis is the process plants use to make food."

↓

4Model inference GPT-4

1000 prompts x variable token length→Generate text outputs using GPT-4→1000 prompts x generated text

"Photosynthesis allows plants to convert sunlight into energy."

↓

5Evaluation

1000 prompts x 2 generated texts→Score outputs by relevance, correctness, and fluency→1000 prompts x 1 best model label

"GPT-4" chosen for prompt 1, "GPT-3.5" for prompt 2

Training Trace - Epoch by Epoch

Loss
0.5 |****
0.4 |******
0.3 |********
0.2 |**********
0.1 |************
     1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.45	0.7	Initial training with moderate loss and accuracy
2	0.35	0.78	Loss decreased, accuracy improved
3	0.28	0.83	Model converging well with steady improvement
4	0.22	0.87	Further loss reduction and accuracy gain
5	0.18	0.9	Training nearing optimal performance

Prediction Trace - 4 Layers

Layer 1: Tokenization

Layer 2: GPT-3.5 inference

Layer 3: GPT-4 inference

Layer 4: Evaluation

Model Quiz - 3 Questions

Which model showed better accuracy during training?

A Both equal

B GPT-3.5

C GPT-4

D Not enough information

Key Insight

Model selection compares outputs from different models on the same input to pick the best one. Training metrics like loss and accuracy help understand model quality, while evaluation of generated text ensures the chosen model meets task needs.