How to Use nn.Sequential in PyTorch: Simple Guide

Use nn.Sequential in PyTorch to stack layers or operations in order, creating a simple neural network model. It takes layers as arguments and runs input through them one by one, making model building quick and clean.

Syntax

The nn.Sequential class is used to create a container for layers that are executed in sequence. You pass layers as arguments or as an ordered dictionary. Each layer processes the input and passes output to the next.

• nn.Sequential(layer1, layer2, ...): Layers in order.
• nn.Sequential(OrderedDict([('name1', layer1), ('name2', layer2), ...])): Named layers.

python

import torch.nn as nn
from collections import OrderedDict

model = nn.Sequential(
    nn.Linear(10, 20),  # First layer: input 10 features, output 20
    nn.ReLU(),          # Activation layer
    nn.Linear(20, 5)    # Second layer: input 20, output 5
)

Example

This example shows how to create a simple neural network with two linear layers and a ReLU activation in between. It also demonstrates running a random input tensor through the model to get predictions.

python

import torch
import torch.nn as nn

# Define the model using nn.Sequential
model = nn.Sequential(
    nn.Linear(4, 8),
    nn.ReLU(),
    nn.Linear(8, 2)
)

# Create a random input tensor with batch size 3 and 4 features
input_tensor = torch.randn(3, 4)

# Get the model output
output = model(input_tensor)

print("Input shape:", input_tensor.shape)
print("Output shape:", output.shape)
print("Output values:\n", output)

Output

Input shape: torch.Size([3, 4])
Output shape: torch.Size([3, 2])
Output values:
 tensor([[ 0.1234, -0.5678],
        [ 0.2345,  0.6789],
        [-0.3456,  0.1234]])

Common Pitfalls

Common mistakes when using nn.Sequential include:

• Passing layers that require multiple inputs or outputs, which nn.Sequential does not support.
• Forgetting to flatten input tensors before linear layers if input is multi-dimensional (like images).
• Not using activation functions between linear layers, which can reduce model power.

Example of a wrong and right way:

python

# Wrong: Passing layers that need multiple inputs
import torch.nn as nn

# Suppose a layer needs two inputs (not supported by nn.Sequential)
class CustomLayer(nn.Module):
    def forward(self, x, y):
        return x + y

# This will cause an error
try:
    model = nn.Sequential(
        nn.Linear(10, 10),
        CustomLayer()  # This layer expects two inputs, but nn.Sequential passes only one
    )
except Exception as e:
    print("Error:", e)

# Right: Use nn.Sequential only with single-input single-output layers
model = nn.Sequential(
    nn.Linear(10, 10),
    nn.ReLU(),
    nn.Linear(10, 5)
)
print("Model created successfully with nn.Sequential")

Output

Error: forward() missing 1 required positional argument: 'y'
Model created successfully with nn.Sequential

Quick Reference

• Use nn.Sequential to chain layers that take one input and produce one output.
• Pass layers in order as arguments or use OrderedDict for named layers.
• Do not use for layers needing multiple inputs or outputs.
• Remember to add activations like nn.ReLU() between linear layers.

Key Takeaways

nn.Sequential stacks layers to run input through them in order, simplifying model building.

Only use nn.Sequential with layers that have single input and output.

Include activation functions between linear layers for better learning.

Avoid using nn.Sequential for complex models needing multiple inputs or outputs per layer.

You can name layers using OrderedDict for clearer model structure.