Optimizers (SGD, Adam) in PyTorch - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create an SGD optimizer for the model parameters with a learning rate of 0.01.

PyTorch

optimizer = torch.optim.[1](model.parameters(), lr=0.01)

Drag options to blanks, or click blank then click option'

ASGD

BAdam

CRMSprop

DAdagrad

Attempts:

3 left

💡 Hint

Common Mistakes

Using Adam instead of SGD when the question asks specifically for SGD.

Misspelling the optimizer name.

The SGD optimizer is created using torch.optim.SGD. It updates parameters using stochastic gradient descent.

2fill in blank

medium

Complete the code to create an Adam optimizer with a learning rate of 0.001.

PyTorch

optimizer = torch.optim.[1](model.parameters(), lr=0.001)

Drag options to blanks, or click blank then click option'

ASGD

BAdadelta

CAdam

DAdamW

Attempts:

3 left

💡 Hint

Common Mistakes

Choosing SGD instead of Adam when adaptive learning is needed.

Confusing Adam with AdamW which is a variant.

The Adam optimizer is created using torch.optim.Adam. It adapts learning rates for each parameter.

3fill in blank

hard

Fix the error in the code to correctly update the optimizer after computing gradients.

PyTorch

loss.backward()
optimizer.[1]()

Drag options to blanks, or click blank then click option'

Astep

Bzero_grad

Cbackward

Dupdate

Attempts:

3 left

💡 Hint

Common Mistakes

Calling 'zero_grad' instead of 'step' after backward pass.

Trying to call 'update' which is not a valid method.

After computing gradients with loss.backward(), call optimizer.step() to update model parameters.

4fill in blank

hard

Fill both blanks to reset gradients and then update parameters correctly.

PyTorch

optimizer.[1]()
loss.backward()
optimizer.[2]()

Drag options to blanks, or click blank then click option'

Azero_grad

Bstep

Cbackward

Dupdate

Attempts:

3 left

💡 Hint

Common Mistakes

Calling 'step' before 'zero_grad'.

Using 'update' which is not a valid optimizer method.

First, clear old gradients with optimizer.zero_grad(). Then compute gradients with loss.backward(). Finally, update parameters with optimizer.step().

5fill in blank

hard

Fill all three blanks to create an Adam optimizer with weight decay, reset gradients, and update parameters.

PyTorch

optimizer = torch.optim.Adam(model.parameters(), lr=[1], weight_decay=[2])
optimizer.[3]()
loss.backward()
optimizer.step()

Drag options to blanks, or click blank then click option'

A0.001

B0.0001

Czero_grad

Dstep

Attempts:

3 left

💡 Hint

Common Mistakes

Using zero_grad after backward pass instead of before.

Confusing weight_decay with learning rate.

The Adam optimizer is created with learning rate 0.001 and weight decay 0.0001. Before backward pass, call optimizer.zero_grad() to clear gradients.