What Generative AI actually is in Prompt Engineering / GenAI - Model Pipeline Trace

Model Pipeline - What Generative AI actually is

Generative AI creates new content like text, images, or music by learning patterns from existing data. It starts with input data, learns features, trains a model to generate similar outputs, and then produces new examples.

Data Flow - 5 Stages

1Data Collection

10000 samples x variable length text→Gather diverse text data from books, articles, and websites→10000 samples x variable length text

"The sun rises in the east."

↓

2Preprocessing

10000 samples x variable length text→Clean text, tokenize words, convert to numbers→10000 samples x 50 tokens

[101, 2023, 2003, 1037, 2742, 102]

↓

3Feature Engineering

10000 samples x 50 tokens→Create embeddings representing word meanings→10000 samples x 50 tokens x 768 features

[[0.12, -0.05, ..., 0.33], [...], ...]

↓

4Model Training

10000 samples x 50 tokens x 768 features→Train a neural network to predict next word→Trained model weights

Model learns to predict 'sun' after 'The'

↓

5Generation

Seed text tokens→Use trained model to generate new text tokens→Generated text sequence

"The sun rises in the morning and shines brightly."

Training Trace - Epoch by Epoch

Loss
2.5 |*****
2.0 |****
1.5 |***
1.0 |**
0.5 |*
    +------------
     1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	2.5	0.30	Model starts learning basic word patterns
2	1.8	0.45	Model improves predicting next words
3	1.3	0.60	Model captures grammar and simple context
4	1.0	0.70	Model generates more coherent sentences
5	0.8	0.78	Model produces fluent and relevant text

Prediction Trace - 5 Layers

Layer 1: Input Tokenization

Layer 2: Embedding Layer

Layer 3: Neural Network Layers

Layer 4: Softmax Activation

Layer 5: Word Selection

Model Quiz - 3 Questions

What does the embedding layer do in Generative AI?

A Cleans the input text

B Generates the final text output

C Turns words into numbers representing their meaning

D Splits text into sentences

Key Insight

Generative AI learns patterns from data to create new content by predicting what comes next. It transforms raw input into meaningful features, trains a model to understand context, and then generates new, coherent outputs.