Generative vs discriminative models in Prompt Engineering / GenAI - Quick Revision & Key Differences

Recall & Review

beginner

What is a generative model in machine learning?

A generative model learns how data is created by modeling the joint probability of inputs and outputs. It can generate new data similar to the training data.

beginner

What is a discriminative model in machine learning?

A discriminative model learns the boundary between classes by modeling the conditional probability of outputs given inputs. It predicts labels for new data.

beginner

Give a real-life example of a generative model.

A generative model is like a chef who learns recipes and can create new dishes. For example, a model that creates new images of faces after learning from many photos.

beginner

Give a real-life example of a discriminative model.

A discriminative model is like a security guard who checks if someone is allowed in or not. For example, a spam filter that decides if an email is spam or not.

beginner

What is the main difference between generative and discriminative models?

Generative models learn how data is made and can create new data. Discriminative models learn to tell classes apart and predict labels.

Which model type can generate new data similar to the training set?

A Both

B Discriminative model

C Neither

D Generative model

Generative models learn the data distribution and can create new samples similar to training data.

Which model focuses on predicting the label given the input?

A Generative model

B Unsupervised model

C Discriminative model

D Reinforcement model

Discriminative models learn the boundary between classes and predict labels from inputs.

Which of these is an example of a discriminative model?

A Logistic Regression

B GAN (Generative Adversarial Network)

C Naive Bayes

D Variational Autoencoder

Logistic Regression is a discriminative model that predicts class labels.

Which model type models the joint probability of inputs and outputs?

A Generative model

B Clustering model

C Regression model

D Discriminative model

Generative models model the joint probability P(inputs, outputs).

What is a key advantage of generative models?

A They are faster to train

B They can create new data samples

C They always have higher accuracy

D They require less data

Generative models can create new data samples similar to the training data.

Explain in your own words the difference between generative and discriminative models.

Think about whether the model creates data or just classifies it.

Give an example of a real-life situation where you would use a generative model and one where you would use a discriminative model.

Consider if the task needs new data or just classification.

Practice

1. Which statement best describes a generative model in machine learning?

easy

A. It only works with labeled data for prediction.

B. It directly learns the boundary between classes for classification.

C. It learns how data is generated and can create new examples.

D. It ignores the data distribution and focuses on accuracy.

Solution

1. Step 1: Understand generative model purpose

   Generative models learn the underlying data distribution to generate new data points similar to the training data.

2. Step 2: Compare with discriminative models

   Discriminative models focus on learning the decision boundary between classes, not on generating data.

3. Final Answer:

   It learns how data is generated and can create new examples. -> Option C

4. Quick Check:

   Generative = create data [OK]

Hint: Generative models create data; discriminative separate classes [OK]

Common Mistakes:

• Confusing generative with discriminative models
• Thinking generative models only classify
• Assuming generative models ignore data distribution

2. Which of the following is the correct way to describe a discriminative model?

easy

A. It models the conditional probability of outputs given inputs.

B. It ignores labels and focuses on data generation.

C. It generates new data points similar to training data.

D. It models the joint probability of inputs and outputs.

Solution

1. Step 1: Define discriminative model behavior

   Discriminative models learn the conditional probability P(output|input), focusing on predicting labels from data.

2. Step 2: Contrast with generative models

   Generative models model the joint probability P(input, output) to generate data, which is not the case here.

3. Final Answer:

   It models the conditional probability of outputs given inputs. -> Option A

4. Quick Check:

   Discriminative = P(output|input) [OK]

Hint: Discriminative models predict labels from inputs [OK]

Common Mistakes:

• Mixing joint and conditional probabilities
• Thinking discriminative models generate data
• Confusing labels with data points

3. Consider the following Python code snippet using scikit-learn:

from sklearn.naive_bayes import GaussianNB
from sklearn.linear_model import LogisticRegression

X_train = [[1, 2], [2, 3], [3, 4], [4, 5]]
y_train = [0, 0, 1, 1]

model = GaussianNB()
model.fit(X_train, y_train)
predictions = model.predict([[2, 3]])
print(predictions)

What will be the output of this code?

medium

A. [1]

B. [0]

C. [0 1]

D. Error due to wrong model usage

Solution

1. Step 1: Identify model type and training data

   GaussianNB is a generative model that learns class distributions. Training data has two classes: 0 and 1.

2. Step 2: Predict class for input [2, 3]

   Input [2, 3] is closer to training points labeled 0 ([1,2],[2,3]) than to those labeled 1, so prediction is class 0.

3. Final Answer:

   [0] -> Option B

4. Quick Check:

   GaussianNB predicts class 0 for [2,3] [OK]

Hint: GaussianNB predicts class based on closest learned distribution [OK]

Common Mistakes:

• Assuming LogisticRegression is used instead
• Expecting multiple classes in output
• Thinking prediction causes error

4. The following code tries to train a discriminative model but has an error:

from sklearn.linear_model import LogisticRegression

X_train = [[1, 2], [2, 3], [3, 4]]
y_train = [0, 1]

model = LogisticRegression()
model.fit(X_train, y_train)

What is the error and how to fix it?

medium

A. Mismatch in number of samples and labels; fix by matching lengths.

B. LogisticRegression requires numeric labels; convert labels to numbers.

C. X_train must be a numpy array; convert list to array.

D. Model.fit() missing parameter; add sample weights.

Solution

1. Step 1: Check training data shapes

   X_train has 3 samples, but y_train has only 2 labels, causing mismatch error.

2. Step 2: Fix label length

   To fix, ensure y_train has 3 labels matching X_train samples, e.g., y_train = [0, 1, 0].

3. Final Answer:

   Mismatch in number of samples and labels; fix by matching lengths. -> Option A

4. Quick Check:

   Samples and labels count must match [OK]

Hint: Check if data and label counts match before training [OK]

Common Mistakes:

• Ignoring label count mismatch
• Assuming LogisticRegression needs label conversion
• Thinking data type causes error

5. You want to build a model that can both classify images of cats and dogs and also generate new realistic images of cats. Which approach should you choose?

hard

A. Use a clustering algorithm to separate and generate images.

B. Use a generative model like a Generative Adversarial Network (GAN) for both tasks.

C. Use a discriminative model like Logistic Regression for both tasks.

D. Use a discriminative model for classification and a generative model for image creation.

Solution

1. Step 1: Identify tasks and suitable models

   Classification is best done by discriminative models that separate classes well. Image generation requires generative models that learn data distribution.

2. Step 2: Combine models for both tasks

   Use a discriminative model for classifying cats vs dogs, and a generative model like GAN to create new cat images.

3. Final Answer:

   Use a discriminative model for classification and a generative model for image creation. -> Option D

4. Quick Check:

   Classification + generation = discriminative + generative [OK]

Hint: Classify with discriminative, generate with generative models [OK]

Common Mistakes:

• Using one model type for both tasks
• Confusing clustering with generation
• Ignoring model strengths for each task