StepLR and MultiStepLR in PyTorch - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is StepLR in PyTorch?

StepLR is a learning rate scheduler that decreases the learning rate by a fixed factor every set number of epochs. It helps the model learn more slowly as training progresses.

intermediate

How does MultiStepLR differ from StepLR?

MultiStepLR decreases the learning rate at specific epochs (called milestones) instead of fixed intervals. This allows more control over when the learning rate changes.

beginner

What parameters do you need to provide to StepLR?

You need to provide the optimizer, step_size (number of epochs between each decrease), and gamma (the factor to multiply the learning rate by).

beginner

What are milestones in MultiStepLR?

Milestones are a list of epoch numbers where the learning rate will be decreased by multiplying with gamma.

beginner

Why use learning rate schedulers like StepLR or MultiStepLR?

They help the model converge better by reducing the learning rate over time, which can improve accuracy and prevent overshooting the best solution.

What does StepLR do to the learning rate?

A Keeps it constant

B Increases it gradually

C Randomly changes it

D Reduces it by a fixed factor every few epochs

StepLR reduces the learning rate by multiplying it by gamma every step_size epochs.

In MultiStepLR, what are milestones?

A Number of batches per epoch

B Epochs where learning rate changes

C Initial learning rates

D Loss values

Milestones are specific epochs where the learning rate is decreased.

Which parameter controls how much the learning rate decreases in StepLR?

A gamma

B step_size

C milestones

D momentum

Gamma is the factor multiplied with the learning rate to reduce it.

If you want to reduce learning rate at epochs 30 and 80, which scheduler is best?

A StepLR

B No scheduler

C MultiStepLR

D ExponentialLR

MultiStepLR lets you specify exact epochs (milestones) to reduce the learning rate.

What is the main benefit of using learning rate schedulers?

A Improve model convergence and accuracy

B Prevent model from learning

C Increase model size

D Faster training without accuracy loss

Schedulers help the model converge better by adjusting learning rate during training.

Explain how StepLR works and what parameters it needs.

Think about a clock ticking every few epochs to reduce learning rate.

Describe the difference between StepLR and MultiStepLR and when you might use each.

Compare fixed steps vs chosen milestones.