requires_grad flag in PyTorch - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does the requires_grad flag do in PyTorch?

It tells PyTorch whether to track operations on a tensor for automatic differentiation. If requires_grad=True, PyTorch records operations to compute gradients later.

beginner

How do you set a tensor to not track gradients in PyTorch?

You set requires_grad=False when creating the tensor or use tensor.detach() to stop tracking gradients.

intermediate

Why might you want to set requires_grad=False for some tensors?

To save memory and computation when you don't need gradients, like for input data or fixed parameters.

beginner

What happens if you perform operations on a tensor with requires_grad=True?

PyTorch builds a computation graph to track those operations, enabling gradient calculation during backpropagation.

beginner

How can you check if a tensor is tracking gradients?

By accessing the tensor's requires_grad attribute, which returns True or False.

What does setting requires_grad=True on a tensor do?

A Converts the tensor to a numpy array

B Prevents the tensor from being used in calculations

C Deletes the tensor after use

D Tracks operations for gradient computation

Setting requires_grad=True tells PyTorch to track operations on the tensor for automatic differentiation.

How can you stop a tensor from tracking gradients after it was created with requires_grad=True?

A Call <code>tensor.backward()</code>

B Set <code>requires_grad=True</code> again

C Use <code>tensor.detach()</code>

D Convert it to a list

tensor.detach() creates a new tensor that does not track gradients.

Why might you want to set requires_grad=False for some tensors?

A To increase the tensor size

B To save memory and speed up computation

C To make the tensor immutable

D To convert the tensor to CPU

Not tracking gradients saves memory and computation when gradients are not needed.

Which attribute tells you if a tensor tracks gradients?

A tensor.requires_grad

B tensor.grad

C tensor.shape

D tensor.dtype

tensor.requires_grad is a boolean indicating if gradients are tracked.

If a tensor has requires_grad=True, what does PyTorch build during operations?

A A computation graph

B A list of numbers

C A new tensor on CPU

D A random tensor

PyTorch builds a computation graph to compute gradients during backpropagation.

Explain the purpose of the requires_grad flag in PyTorch and how it affects tensor operations.

Think about how PyTorch knows which tensors to compute gradients for.

Describe scenarios when you would set requires_grad=False for a tensor in a machine learning model.

Consider parts of the model that do not learn or update.