Generative vs discriminative models in Prompt Engineering / GenAI - Model Approaches Compared

Model Pipeline - Generative vs discriminative models

This pipeline compares two types of models: generative models that learn how data is made, and discriminative models that learn to tell categories apart.

Data Flow - 5 Stages

1Input Data

1000 rows x 3 columns→Raw data with features and labels→1000 rows x 3 columns

[Feature1=5.1, Feature2=3.5, Label=ClassA]

↓

2Generative Model Training

1000 rows x 3 columns→Model learns joint probability of features and labels→Model parameters representing data distribution

Learned means and variances for each class

↓

3Discriminative Model Training

1000 rows x 3 columns→Model learns decision boundary between classes→Model parameters for classification

Weights for logistic regression separating classes

↓

4Prediction with Generative Model

1 row x 2 columns→Calculate probability of each class given features→Class probabilities

[ClassA: 0.7, ClassB: 0.3]

↓

5Prediction with Discriminative Model

1 row x 2 columns→Calculate class probability directly→Class probabilities

[ClassA: 0.8, ClassB: 0.2]

Training Trace - Epoch by Epoch

Loss
0.7 |*       
0.6 | **     
0.5 |  ***   
0.4 |   **** 
0.3 |    *****
    +---------
     1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.65	0.60	Both models start with moderate loss and accuracy.
2	0.50	0.72	Loss decreases and accuracy improves as models learn.
3	0.40	0.80	Clear improvement; discriminative model may learn faster.
4	0.35	0.85	Loss continues to decrease; accuracy rises steadily.
5	0.30	0.88	Models converge with good accuracy.

Prediction Trace - 4 Layers

Layer 1: Input features

Layer 2: Generative model calculates likelihood

Layer 3: Discriminative model calculates probability

Layer 4: Final decision

Model Quiz - 3 Questions

Which model type learns how data is generated?

A Discriminative model

B Generative model

C Neither model

D Both models

Key Insight

Generative models learn the full data distribution and can generate data, while discriminative models focus on drawing boundaries to classify data. Both improve with training as loss decreases and accuracy increases.