Sequence classification in PyTorch

Introduction

Sequence classification helps us teach a computer to understand and label a series of items, like words in a sentence or steps in a process.

To decide if an email is spam or not based on its words.

To recognize the sentiment (happy, sad) in a movie review.

To classify DNA sequences in biology.

To detect commands from spoken words in voice assistants.

To categorize news articles by topic from their text.

Syntax

PyTorch

import torch
import torch.nn as nn

class SequenceClassifier(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super(SequenceClassifier, self).__init__()
        self.rnn = nn.RNN(input_size, hidden_size, batch_first=True)
        self.fc = nn.Linear(hidden_size, num_classes)

    def forward(self, x):
        out, _ = self.rnn(x)
        out = out[:, -1, :]
        out = self.fc(out)
        return out

The input to the model is a sequence of vectors (like word embeddings).

The RNN processes the sequence step by step, and we use the last output to classify the whole sequence.

Examples

This creates a model that takes sequences where each item has 10 features, uses 20 hidden units in the RNN, and classifies into 3 classes.

PyTorch

model = SequenceClassifier(input_size=10, hidden_size=20, num_classes=3)

Here, we pass a batch of 5 sequences, each 7 steps long, with 10 features per step. The output will have shape (5, 3) for 3 classes.

PyTorch

output = model(torch.randn(5, 7, 10))

Sample Model

This program creates a simple RNN model to classify sequences into two classes. It trains on random data for 5 steps and prints loss and accuracy.

PyTorch

import torch
import torch.nn as nn
import torch.optim as optim

# Define the model
class SequenceClassifier(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super(SequenceClassifier, self).__init__()
        self.rnn = nn.RNN(input_size, hidden_size, batch_first=True)
        self.fc = nn.Linear(hidden_size, num_classes)

    def forward(self, x):
        out, _ = self.rnn(x)
        out = out[:, -1, :]
        out = self.fc(out)
        return out

# Parameters
input_size = 5
hidden_size = 10
num_classes = 2
batch_size = 4
seq_length = 6

# Create model, loss, optimizer
model = SequenceClassifier(input_size, hidden_size, num_classes)
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.01)

# Dummy data: batch of 4 sequences, each 6 steps, each step 5 features
inputs = torch.randn(batch_size, seq_length, input_size)
# Labels: 4 labels for the batch
labels = torch.tensor([0, 1, 0, 1])

# Training loop for 5 epochs
for epoch in range(5):
    optimizer.zero_grad()
    outputs = model(inputs)
    loss = criterion(outputs, labels)
    loss.backward()
    optimizer.step()
    _, predicted = torch.max(outputs, 1)
    accuracy = (predicted == labels).float().mean()
    print(f"Epoch {epoch+1}, Loss: {loss.item():.4f}, Accuracy: {accuracy.item():.4f}")

Important Notes

Sequence length and batch size can vary depending on your data.

RNNs can be replaced with LSTM or GRU for better performance on longer sequences.

Always check that your input data shape matches the model's expected input.

Summary

Sequence classification labels whole sequences, not just single items.

RNNs process sequences step by step and help capture order information.

The last output of the RNN is used to predict the class of the entire sequence.