How to Count Model Parameters in PyTorch Easily

To count model parameters in PyTorch, use sum(p.numel() for p in model.parameters()) which sums all elements in each parameter tensor. For only trainable parameters, add a condition: sum(p.numel() for p in model.parameters() if p.requires_grad).

Syntax

Use model.parameters() to get all parameters of a PyTorch model. Then, p.numel() returns the number of elements in each parameter tensor. Summing these gives the total count.

• model.parameters(): iterator over model parameters
• p.numel(): number of elements in parameter tensor p
• p.requires_grad: boolean indicating if parameter is trainable

python

total_params = sum(p.numel() for p in model.parameters())
trainable_params = sum(p.numel() for p in model.parameters() if p.requires_grad)

Example

This example creates a simple neural network and counts both total and trainable parameters.

python

import torch
import torch.nn as nn

class SimpleNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(10, 5)
        self.fc2 = nn.Linear(5, 2)

    def forward(self, x):
        x = self.fc1(x)
        x = self.fc2(x)
        return x

model = SimpleNet()

total_params = sum(p.numel() for p in model.parameters())
trainable_params = sum(p.numel() for p in model.parameters() if p.requires_grad)

print(f"Total parameters: {total_params}")
print(f"Trainable parameters: {trainable_params}")

Output

Total parameters: 72
Trainable parameters: 72

Common Pitfalls

One common mistake is counting parameters without checking if they are trainable, which can be misleading if some layers are frozen. Another is forgetting to call model.parameters() and trying to count parameters directly from layers.

Also, summing len(p) instead of p.numel() only counts the first dimension, giving wrong results.

python

wrong_count = sum(len(p) for p in model.parameters())  # Incorrect
correct_count = sum(p.numel() for p in model.parameters())  # Correct

Quick Reference

Remember these tips when counting parameters:

• Use p.numel() to count all elements in a parameter tensor.
• Filter with p.requires_grad to count only trainable parameters.
• Use model.parameters() to access all parameters.

Key Takeaways

Use sum of p.numel() over model.parameters() to count total parameters.

Filter parameters with p.requires_grad to count only trainable ones.

Avoid using len(p) as it counts only the first dimension, not total elements.

model.parameters() provides all model parameters as tensors.

Counting parameters helps understand model size and complexity.