Gradient access (.grad) in PyTorch - ML Experiment: Train & Evaluate

Experiment - Gradient access (.grad)

Problem:You have a simple linear model trained on a small dataset. You want to understand how gradients are computed and accessed during backpropagation.

Current Metrics:Training loss: 0.25, Validation loss: 0.27

Issue:You do not see or understand how to access the gradients of model parameters after backpropagation.

Your Task

Access and print the gradients of the model parameters after one backward pass to understand how .grad works in PyTorch.

Do not change the model architecture or dataset.

Only add code to access and print gradients after loss.backward().

Hint 1

Hint 2

Hint 3

Solution

PyTorch

import torch
import torch.nn as nn
import torch.optim as optim

# Simple dataset
X = torch.tensor([[1.0], [2.0], [3.0], [4.0]])
y = torch.tensor([[2.0], [4.0], [6.0], [8.0]])

# Simple linear model
model = nn.Linear(1, 1)

# Loss and optimizer
criterion = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=0.01)

# Forward pass
outputs = model(X)
loss = criterion(outputs, y)

# Backward pass
optimizer.zero_grad()
loss.backward()

# Access and print gradients
for name, param in model.named_parameters():
    if param.grad is not None:
        print(f"Gradient of {name}: {param.grad}")
    else:
        print(f"No gradient for {name}")

Added code to print gradients of each model parameter after loss.backward()

Used model.named_parameters() to get parameter names and gradients

Ensured optimizer.zero_grad() is called before backward to clear old gradients

Results Interpretation

Before: No access to gradients, so no insight into parameter updates.

After: Printed gradients for weights and bias show how much each parameter will change during optimization.

The .grad attribute in PyTorch stores the gradient of each parameter after backpropagation. Accessing .grad helps understand and debug training.

Bonus Experiment

Try zeroing gradients manually and observe what happens if you forget to call optimizer.zero_grad() before backward.

💡 Hint

Run backward twice without zeroing gradients and print .grad to see accumulation.