Bidirectional RNNs in PyTorch

Introduction

Bidirectional RNNs help a model understand information from both past and future in a sequence. This makes predictions better when context from both sides matters.

When reading a sentence and you want to understand each word using both previous and next words.

When analyzing time series data where future and past points influence the current value.

When processing audio signals where sounds before and after affect recognition.

When working with DNA sequences where context on both sides helps identify patterns.

Syntax

PyTorch

torch.nn.RNN(input_size, hidden_size, num_layers=1, bidirectional=True)

Set bidirectional=True to make the RNN read input forwards and backwards.

The output size doubles because it combines forward and backward passes.

Examples

This creates a bidirectional RNN with input size 10 and hidden size 20.

PyTorch

rnn = torch.nn.RNN(input_size=10, hidden_size=20, bidirectional=True)

This creates a 2-layer bidirectional RNN for deeper sequence understanding.

PyTorch

rnn = torch.nn.RNN(input_size=5, hidden_size=15, num_layers=2, bidirectional=True)

Sample Model

This code creates a bidirectional RNN and runs a random input through it. The output shape shows the combined forward and backward hidden states for each step. The hidden state contains the last hidden states from both directions.

PyTorch

import torch
import torch.nn as nn

# Parameters
input_size = 3
hidden_size = 4
seq_len = 5
batch_size = 2

# Create bidirectional RNN
rnn = nn.RNN(input_size, hidden_size, bidirectional=True, batch_first=True)

# Random input: batch_size sequences, each with seq_len steps, each step with input_size features
inputs = torch.randn(batch_size, seq_len, input_size)

# Forward pass
outputs, hidden = rnn(inputs)

print('Output shape:', outputs.shape)
print('Output:', outputs)
print('Hidden shape:', hidden.shape)
print('Hidden:', hidden)

Important Notes

The output size is hidden_size * 2 because it combines forward and backward states.

Use batch_first=True to have input shape as (batch, seq_len, features) for easier handling.

Hidden state shape is (num_layers * 2, batch, hidden_size) because of two directions.

Summary

Bidirectional RNNs read sequences forwards and backwards to capture full context.

They double the hidden size in output by combining two directions.

Useful when both past and future information matter for predictions.