requires_grad flag in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of requires_grad flag after tensor operations?

Consider the following PyTorch code snippet. What will be the value of z.requires_grad after the operations?

PyTorch

import torch
x = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
y = x * 2
y = y.detach()
z = y + 1

A None

B True

C Raises an error

D False

💡 Hint

Think about what detach() does to the computation graph and the requires_grad flag.

The detach() method creates a new tensor that shares data but does not require gradients. Therefore, y.requires_grad becomes False, and so does z.requires_grad because it is computed from y.

❓ Model Choice

intermediate

Which tensor requires gradients for training a model?

You want to train a simple linear model in PyTorch. Which tensor should have requires_grad=True to update its values during training?

A Input data tensor

B Model parameters tensor

C Target labels tensor

D Random noise tensor

💡 Hint

Which tensors need gradients to update weights during backpropagation?

Model parameters need requires_grad=True so gradients are computed and weights updated. Input data and target labels do not require gradients.

❓ Hyperparameter

advanced

How does setting requires_grad=False affect training speed?

What is the effect of setting requires_grad=False on some model parameters during training?

A Causes training to fail with an error

B Slows down training because gradients are still computed

C Speeds up training by skipping gradient computation for those parameters

D Has no effect on training speed

💡 Hint

Think about what happens when gradients are not computed for some tensors.

Setting requires_grad=False tells PyTorch not to compute gradients for those tensors, reducing computation and speeding up training.

🔧 Debug

advanced

Why does this code raise an error related to requires_grad?

Examine the code below. Why does the backward call raise an error?

PyTorch

import torch
x = torch.tensor([1.0, 2.0, 3.0])
y = x * 2
z = y.sum()
z.backward()

A Because x does not have requires_grad=True

B Because y is not a tensor

C Because sum() cannot be used before backward()

D Because backward() requires an argument

💡 Hint

Check if the tensor you want gradients for has requires_grad=True.

The tensor x was created without requires_grad=True, so PyTorch does not track operations for gradients. Calling backward() on z raises an error.

🧠 Conceptual

expert

What happens to requires_grad flag after in-place operations?

In PyTorch, if you perform an in-place operation on a tensor with requires_grad=True, what is the effect on the requires_grad flag and the computation graph?

A The <code>requires_grad</code> flag remains True, but the computation graph may be corrupted causing errors during backward

B The <code>requires_grad</code> flag automatically switches to False

C The tensor becomes detached from the graph but keeps <code>requires_grad=True</code>

D In-place operations are not allowed on tensors with <code>requires_grad=True</code> and raise an immediate error

💡 Hint

Consider how in-place changes affect gradient tracking and graph integrity.

In-place operations keep requires_grad=True but can corrupt the computation graph, leading to runtime errors during backpropagation. PyTorch warns about this but does not automatically disable gradient tracking.