Why Fine-tuning approach in Computer Vision? - Purpose & Use Cases

The Big Idea

What if you could teach a computer new skills in hours instead of weeks?

The Scenario

Imagine you want to teach a computer to recognize different types of flowers. Doing this from scratch means collecting thousands of pictures, labeling them, and training a model for days or weeks.

The Problem

Training a model from zero is slow and needs a lot of data. It's easy to make mistakes, and the computer might not learn well if the data is small or messy.

The Solution

Fine-tuning lets you start with a model already trained on many images. You only adjust it a little with your flower pictures, making learning faster and more accurate.

Before vs After

✗ Before

model = create_model()
model.train(flower_images, flower_labels)

✓ After

base_model = load_pretrained_model()
model = fine_tune(base_model, flower_images, flower_labels)

What It Enables

Fine-tuning unlocks quick, accurate learning on new tasks with less data and effort.

Real Life Example

A company uses fine-tuning to adapt a general image model to spot defects in their products, saving time and improving quality.

Key Takeaways

Training from scratch is slow and needs lots of data.

Fine-tuning uses existing knowledge to learn faster.

This approach works well even with small, specific datasets.

Practice

1. What is the main purpose of fine-tuning a pre-trained computer vision model?

easy

A. To adapt the model to a new task using less data and time

B. To train a model from scratch with a large dataset

C. To increase the size of the model for better accuracy

D. To remove layers from the model to make it smaller

Solution

1. Step 1: Understand fine-tuning concept

   Fine-tuning means starting from a model already trained on a related task.

2. Step 2: Identify the benefit

   This approach saves time and data by reusing learned features for a new task.

3. Final Answer:

   To adapt the model to a new task using less data and time -> Option A

4. Quick Check:

   Fine-tuning = adapt pre-trained model fast [OK]

Hint: Fine-tuning means reusing a model to learn new tasks faster [OK]

Common Mistakes:

• Thinking fine-tuning trains from scratch
• Assuming fine-tuning always increases model size
• Confusing fine-tuning with pruning layers

2. Which code snippet correctly freezes the layers of a PyTorch model before fine-tuning?

easy

A. for param in model.parameters(): param.requires_grad = False

B. model.freeze_layers()

C. model.trainable = False

D. for layer in model.layers: layer.trainable = True

Solution

1. Step 1: Recall PyTorch freezing syntax

   In PyTorch, freezing means setting requires_grad = False for parameters.

2. Step 2: Match code to syntax

   for param in model.parameters(): param.requires_grad = False correctly loops over parameters and disables gradient updates.

3. Final Answer:

   for param in model.parameters(): param.requires_grad = False -> Option A

4. Quick Check:

   Freeze layers = requires_grad False [OK]

Hint: Freeze layers by setting requires_grad = False in PyTorch [OK]

Common Mistakes:

• Using non-existent methods like freeze_layers()
• Setting model.trainable instead of parameters
• Confusing trainable True/False for freezing

3. Given this PyTorch code snippet for fine-tuning, what will be the output of print(sum(p.requires_grad for p in model.parameters())) after freezing layers?

medium

A. Raises an error

B. Number of all model parameters

C. 0

D. Number of unfrozen parameters

Solution

1. Step 1: Understand freezing effect on requires_grad

   Freezing sets requires_grad = False for all parameters.

2. Step 2: Calculate sum of requires_grad flags

   Since all are False, sum counts zero True values.

3. Final Answer:

   0 -> Option C

4. Quick Check:

   All frozen means requires_grad sum = 0 [OK]

Hint: Frozen layers have requires_grad = False, sum is zero [OK]

Common Mistakes:

• Assuming sum counts total parameters
• Thinking sum counts unfrozen parameters without freezing
• Expecting an error from requires_grad attribute

4. You tried fine-tuning but the model's accuracy did not improve. Which mistake could cause this?

medium

A. Using a pre-trained model instead of training from scratch

B. Freezing all layers and not unfreezing any

C. Adding more layers without training them

D. Using a very high learning rate during fine-tuning

Solution

1. Step 1: Identify learning rate impact

   A very high learning rate can cause unstable training and no improvement.

2. Step 2: Evaluate other options

   Freezing all layers prevents learning but usually keeps baseline accuracy; pre-trained models help; adding untrained layers alone doesn't prevent improvement if trained.

3. Final Answer:

   Using a very high learning rate during fine-tuning -> Option D

4. Quick Check:

   High learning rate = no improvement [OK]

Hint: Use smaller learning rates for fine-tuning to improve accuracy [OK]

Common Mistakes:

• Ignoring learning rate effects
• Assuming freezing all layers always improves
• Thinking training from scratch is better always

5. You want to fine-tune a pre-trained CNN for a new image classification task with 5 classes. Which sequence of steps is best practice?

hard

A. Train entire model from scratch with random weights for 5 classes

B. Freeze all layers, replace final layer with 5 outputs, train only final layer, then unfreeze some layers and fine-tune with low learning rate

C. Replace final layer with 5 outputs and train all layers at once with high learning rate

D. Freeze final layer, train earlier layers only, then unfreeze final layer

Solution

1. Step 1: Replace final layer for new classes

   Adjust output layer to match 5 classes for the new task.

2. Step 2: Freeze old layers and train new layer first

   This preserves learned features and trains new output layer quickly.

3. Step 3: Unfreeze some layers and fine-tune with low learning rate

   This improves model performance by adapting features carefully without large updates.

4. Final Answer:

   Freeze all layers, replace final layer with 5 outputs, train only final layer, then unfreeze some layers and fine-tune with low learning rate -> Option B

5. Quick Check:

   Stepwise fine-tuning with low LR = best practice [OK]

Hint: Freeze, replace output, train new layer, then unfreeze and fine-tune [OK]

Common Mistakes:

• Training all layers at once with high learning rate
• Training from scratch ignoring pre-trained weights
• Freezing final layer instead of earlier layers