Gradient access (.grad) in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of accessing .grad after backward()?

Consider the following PyTorch code snippet. What will be the value of x.grad after running it?

PyTorch

import torch
x = torch.tensor(2.0, requires_grad=True)
y = x ** 3
loss = y
loss.backward()
print(x.grad.item())

A 6.0

B 8.0

C 12.0

D None

💡 Hint

Recall the derivative of x^3 is 3*x^2.

The gradient of y = x^3 with respect to x is 3*x^2. At x=2, this is 3*4=12. So x.grad will be 12.0 after backward().

❓ Model Choice

intermediate

Which tensor will have .grad populated after backward()?

Given the following tensors, which one will have its .grad attribute populated after calling loss.backward()?

PyTorch

import torch
x = torch.tensor(1.0, requires_grad=True)
y = torch.tensor(2.0)
z = x * y
loss = z
loss.backward()

A x

B y

C z

D loss

💡 Hint

Only tensors with requires_grad=True track gradients.

Only x has requires_grad=True, so only x.grad will be populated after backward().

❓ Hyperparameter

advanced

What happens if you call backward() twice without zeroing gradients?

In PyTorch, if you call loss.backward() twice on the same graph without zeroing gradients, what will happen to x.grad?

PyTorch

import torch
x = torch.tensor(3.0, requires_grad=True)
y = x ** 2
loss = y
loss.backward()
loss.backward()
print(x.grad.item())

A The gradient will be zero

B The gradient will be reset to the last backward call (9.0)

C An error will be raised

D The gradient will be doubled (12.0)

💡 Hint

Gradients accumulate by default in PyTorch.

Calling backward() twice accumulates gradients. The derivative of x^2 at x=3 is 2*x=6. After two backward calls, x.grad is 6 + 6 = 12.0.

🔧 Debug

advanced

Why is x.grad None after backward()?

Look at this code. Why is x.grad None after calling loss.backward()?

PyTorch

import torch
x = torch.tensor(4.0)
y = x ** 2
loss = y
loss.backward()
print(x.grad)

A x does not have requires_grad=True

B x.grad was manually set to None

C loss.backward() was not called

D loss is not a scalar

💡 Hint

Check if the tensor tracks gradients.

Since x was created without requires_grad=True, PyTorch does not track operations on it, so x.grad remains None.

🧠 Conceptual

expert

What is the shape and content of .grad for a tensor with multiple elements?

Given the code below, what will be the shape and values of x.grad after loss.backward()?

PyTorch

import torch
x = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
y = x ** 2
loss = y.sum()
loss.backward()
print(x.grad)

A Tensor(6.0) scalar

B Tensor([2.0, 4.0, 6.0]) with shape (3,)

C Tensor([1.0, 2.0, 3.0]) with shape (3,)

D None

💡 Hint

Derivative of x^2 is 2*x element-wise.

The gradient of sum(x^2) w.r.t x is 2*x element-wise. So x.grad is [2,4,6] with shape (3,).