GPT family overview in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding GPT Model Sizes

Which GPT model in the family is known for having the largest number of parameters?

A GPT-1

B GPT-3

C GPT-2

D GPT-Neo

💡 Hint

Think about the model that marked a big jump in size and capability.

GPT-3 is the largest among the original GPT family models, with 175 billion parameters, much larger than GPT-2 and GPT-1.

🧠 Conceptual

intermediate

Training Data Differences

What is a key difference in the training data between GPT-2 and GPT-3?

A GPT-3 was trained only on code, GPT-2 on natural language

B GPT-2 used only books, GPT-3 used only websites

C GPT-3 was trained on a much larger and more diverse dataset than GPT-2

D GPT-2 used supervised learning, GPT-3 used reinforcement learning

💡 Hint

Consider the scale and variety of data used.

GPT-3 was trained on a significantly larger and more diverse dataset, including Common Crawl, books, and other sources, compared to GPT-2.

❓ Model Choice

advanced

Choosing GPT Model for Few-Shot Learning

Which GPT model is best suited for few-shot learning tasks without fine-tuning?

A GPT-3

B GPT-1

C GPT-2

D GPT-2 Small

💡 Hint

Look for the model known for strong few-shot capabilities.

GPT-3 demonstrated strong few-shot learning abilities, allowing it to perform tasks with just a few examples without retraining.

❓ Metrics

advanced

Evaluating GPT Model Performance

Which metric is commonly used to evaluate the language modeling performance of GPT models?

A Perplexity

B Mean Squared Error

C F1 Score

D Accuracy

💡 Hint

This metric measures how well a model predicts the next word.

Perplexity measures how well a language model predicts a sample; lower perplexity means better performance.

🔧 Debug

expert

Identifying Cause of GPT Model Output Bias

A GPT model consistently generates biased or inappropriate text. What is the most likely cause?

A The model architecture is too small

B The model was trained for too many epochs

C The optimizer used was incorrect

D The training data contains biased or unfiltered content

💡 Hint

Think about what influences the model's learned behavior most.

Bias in GPT outputs usually comes from biased or unfiltered training data, which the model learns from.