Pre-training and fine-tuning concept in Prompt Engineering / GenAI - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is pre-training in machine learning?

Pre-training is the process where a model learns general patterns from a large dataset before being trained on a specific task. It's like learning the basics first before focusing on details.

beginner

What does fine-tuning mean in AI model training?

Fine-tuning means adjusting a pre-trained model on a smaller, specific dataset to make it perform well on a particular task, like customizing a general skill to a special job.

intermediate

Why do we use pre-training before fine-tuning?

Pre-training helps the model learn general knowledge, which saves time and data when fine-tuning. It’s like learning the alphabet before writing a story.

beginner

Give a real-life example of pre-training and fine-tuning.

Imagine learning to drive (pre-training) before learning to drive a race car (fine-tuning). You first learn general driving skills, then adjust to the special car’s needs.

intermediate

What is a benefit of fine-tuning a pre-trained model?

Fine-tuning allows the model to perform well on a new task with less data and time, making it faster and cheaper to get good results.

What is the main goal of pre-training a model?

A To learn general patterns from a large dataset

B To train on a small specific task dataset

C To test the model's accuracy

D To delete unnecessary data

Pre-training helps the model learn general knowledge from a large dataset before focusing on specific tasks.

Fine-tuning is best described as:

A Training a model from scratch

B Adjusting a pre-trained model on a specific task

C Collecting more data

D Evaluating model performance

Fine-tuning means adapting a pre-trained model to perform well on a particular task.

Which of these is a benefit of pre-training?

A Makes the model slower

B Requires more data for each new task

C Saves time when training on new tasks

D Removes the need for fine-tuning

Pre-training saves time and data when adapting the model to new tasks.

What is an example of fine-tuning?

A Adjusting a general language model to understand medical terms

B Learning to read before writing

C Training a model on a large dataset

D Deleting old training data

Fine-tuning customizes a general model to perform well on a specific domain like medical language.

Why is fine-tuning important after pre-training?

A It removes errors from pre-training

B It helps the model forget old knowledge

C It increases the model size

D It customizes the model for a specific task

Fine-tuning adapts the pre-trained model to perform well on a particular task.

Explain in your own words what pre-training and fine-tuning mean and how they work together.

Think about learning general skills first, then specializing.

Describe a simple example from daily life that helps you understand why pre-training and fine-tuning are useful.

Consider learning a basic skill before a specialized one.