Learning rate selection in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Learning rate selection

Which metric matters for Learning Rate Selection and WHY

When choosing a learning rate, the key metric to watch is the training loss and validation loss over time. These show how well the model is learning. A good learning rate helps the loss go down steadily without jumping around or getting stuck.

We also look at accuracy on validation data to see if the model is improving in making correct predictions. If accuracy improves smoothly, the learning rate is likely good.

Why? Because the learning rate controls how big each step is when the model learns. Too big, and the model jumps past good answers. Too small, and learning is very slow or stuck.

Confusion Matrix or Equivalent Visualization

Learning rate itself does not have a confusion matrix, but its effect shows in training curves:

Epoch | Training Loss | Validation Loss | Validation Accuracy
------------------------------------------------------------
  1   |     1.2      |      1.3       |       45%
  2   |     0.9      |      1.0       |       55%
  3   |     0.7      |      0.8       |       65%
  ... |     ...      |      ...       |       ...

Good learning rate: smooth, steady loss decrease and accuracy increase.

Too high learning rate: loss jumps up and down or diverges.

Too low learning rate: loss decreases very slowly or plateaus.

Tradeoff: Learning Rate Size vs Model Performance

High learning rate: Fast learning but risk of missing the best solution. Loss may bounce or increase.

Low learning rate: Stable learning but very slow progress. May get stuck in a bad solution.

Example: Imagine trying to find the bottom of a valley blindfolded.

• Big steps (high learning rate) might make you overshoot the bottom repeatedly.
• Small steps (low learning rate) make you move slowly but carefully.

Choosing the right step size helps you reach the bottom efficiently.

What Good vs Bad Learning Rate Looks Like

Good learning rate:

• Training and validation loss decrease steadily.
• Validation accuracy improves smoothly.
• No sudden jumps or spikes in loss.

Bad learning rate (too high):

• Loss values jump up and down or increase.
• Validation accuracy fluctuates or drops.
• Training may fail to converge.

Bad learning rate (too low):

• Loss decreases very slowly or plateaus.
• Accuracy improves very slowly or not at all.
• Training takes too long.

Common Pitfalls in Learning Rate Selection

• Ignoring validation loss: Only watching training loss can hide overfitting or poor generalization.
• Using a fixed learning rate: Sometimes a learning rate schedule or decay helps improve training.
• Too large initial learning rate: Can cause training to diverge immediately.
• Too small learning rate: Training may get stuck in local minima or take too long.
• Not tuning learning rate with batch size: Larger batch sizes often need different learning rates.

Self Check

Your model training shows loss jumping up and down wildly and validation accuracy is not improving after many epochs. You are using a learning rate of 0.1. Is this good?

Answer: No, this learning rate is likely too high. The jumps in loss and no accuracy improvement mean the model is not learning well. Try lowering the learning rate to get smoother, steady training progress.

Key Result

Learning rate affects training loss and validation accuracy curves; a good rate leads to steady loss decrease and accuracy improvement.