Learning rate selection in Computer Vision - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why is choosing the right learning rate important?

Imagine you are teaching a robot to recognize cats in photos. You adjust how fast it learns by setting a learning rate. What happens if the learning rate is too high?

A The robot will always get perfect results immediately.

B The robot learns too slowly and takes a long time to improve.

C The robot might miss the correct answer because it jumps too much and never settles.

D The robot ignores the photos and guesses randomly.

💡 Hint

Think about how big steps affect finding the right path.

A high learning rate causes the model to take big steps in learning, which can overshoot the best solution and fail to settle, causing unstable training.

❓ Predict Output

intermediate

Output of training loss with different learning rates

Look at the training loss values after 3 epochs with different learning rates. Which learning rate likely caused the loss to increase instead of decrease?

Computer Vision

losses = {
    0.001: [0.9, 0.7, 0.5],
    0.1: [0.9, 1.2, 1.5],
    0.01: [0.9, 0.6, 0.4],
    0.0001: [0.9, 0.85, 0.8]
}

# Which learning rate caused loss to increase?

A 0.1

B 0.001

C 0.01

D 0.0001

💡 Hint

Look for loss values that get bigger over time.

With learning rate 0.1, loss increases from 0.9 to 1.5, indicating unstable training due to too high learning rate.

❓ Hyperparameter

advanced

Choosing learning rate for a convolutional neural network

You train a convolutional neural network on images. You want to try these learning rates: 0.1, 0.01, 0.001, 0.0001. Which learning rate is most likely to cause the model to converge smoothly without overshooting?

A 0.01

B 1.0

C 0.1

D 0.0001

💡 Hint

Think about typical learning rates used in image models.

0.01 is a common starting learning rate for CNNs that balances speed and stability.

❓ Metrics

advanced

Effect of learning rate on validation accuracy

After training a model with different learning rates, you get these validation accuracies:

• 0.1: 60%
• 0.01: 85%
• 0.001: 80%
• 0.0001: 50%

Which learning rate likely caused underfitting?

A 0.1

B 0.01

C 0.001

D 0.0001

💡 Hint

Underfitting means the model learns too slowly and performs poorly.

Learning rate 0.0001 is too small, causing slow learning and low accuracy, a sign of underfitting.

🔧 Debug

expert

Diagnosing training failure due to learning rate

You train a deep neural network but notice the training loss stays very high and does not improve. You suspect the learning rate is the cause. Which of these symptoms best supports that the learning rate is too high?

A Training loss decreases smoothly over epochs.

B Training loss fluctuates wildly and sometimes increases sharply.

C Training loss decreases very slowly and plateaus early.

D Training accuracy is perfect but validation accuracy is low.

💡 Hint

Think about what happens when steps are too big during learning.

Wild fluctuations and sharp increases in loss indicate the learning rate is too high, causing unstable training.