nn.RNN layer in PyTorch

Introduction

The nn.RNN layer helps a computer remember information from a sequence, like words in a sentence, so it can understand patterns over time.

When you want to predict the next word in a sentence.

When analyzing time series data like stock prices.

When processing audio signals to recognize speech.

When working with sequences of sensor data from devices.

Syntax

PyTorch

torch.nn.RNN(input_size, hidden_size, num_layers=1, nonlinearity='tanh', batch_first=False, dropout=0, bidirectional=False)

input_size is the number of features in each input step.

hidden_size is how many features the RNN remembers at each step.

Examples

Creates a simple RNN with input size 10 and hidden size 20.

PyTorch

rnn = torch.nn.RNN(input_size=10, hidden_size=20)

Creates a 2-layer RNN where input and output tensors have batch as the first dimension.

PyTorch

rnn = torch.nn.RNN(5, 15, num_layers=2, batch_first=True)

Creates a bidirectional RNN using ReLU activation.

PyTorch

rnn = torch.nn.RNN(8, 12, nonlinearity='relu', bidirectional=True)

Sample Model

This code creates a simple RNN layer and passes a random input sequence through it. It prints the output and the final hidden state shapes and values.

PyTorch

import torch
import torch.nn as nn

# Create an RNN layer
rnn = nn.RNN(input_size=3, hidden_size=5, num_layers=1, batch_first=True)

# Sample input: batch size 2, sequence length 4, input size 3
input_seq = torch.randn(2, 4, 3)

# Initial hidden state: num_layers=1, batch=2, hidden_size=5
h0 = torch.zeros(1, 2, 5)

# Forward pass
output, hn = rnn(input_seq, h0)

print('Output shape:', output.shape)
print('Output:', output)
print('Hidden state shape:', hn.shape)
print('Hidden state:', hn)

Important Notes

The RNN output has shape (batch, seq_len, hidden_size) if batch_first=True.

The hidden state contains the last output for each sequence in the batch.

Use batch_first=True to make input and output tensors easier to work with (batch first).

Summary

nn.RNN layer processes sequences step by step, remembering past information.

It takes input size and hidden size to set up the memory and input features.

Outputs include the full sequence output and the last hidden state.