Bidirectional LSTM in NLP

Introduction

Bidirectional LSTM helps a model understand information from both past and future in a sequence. This makes it better at tasks like language understanding.

When you want to understand the meaning of a sentence by looking at words before and after each word.

When analyzing time series data where future and past values both matter.

When building chatbots that need context from the whole conversation.

When doing speech recognition to better predict words based on surrounding sounds.

When working on text classification where context from both directions improves accuracy.

Syntax

NLP

from tensorflow.keras.layers import Bidirectional, LSTM

bidirectional_layer = Bidirectional(LSTM(units))

The Bidirectional wrapper takes a recurrent layer like LSTM and runs it forwards and backwards.

The units parameter sets how many memory cells the LSTM has.

Examples

This creates a bidirectional LSTM with 64 memory units in each direction.

NLP

Bidirectional(LSTM(64))

This returns the output for each time step, useful for stacking layers.

NLP

Bidirectional(LSTM(32, return_sequences=True))

This adds dropout to reduce overfitting during training.

NLP

Bidirectional(LSTM(128, dropout=0.2))

Sample Model

This example builds a simple model with a bidirectional LSTM layer to classify sequences. It trains on random data and prints predictions.

NLP

import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, Bidirectional, LSTM, Dense

# Sample data: 5 sequences, each with 10 words (integers)
x_train = np.random.randint(1, 1000, (5, 10))
y_train = np.array([0, 1, 0, 1, 0])  # Binary labels

model = Sequential([
    Embedding(input_dim=1000, output_dim=16, input_length=10),
    Bidirectional(LSTM(8)),
    Dense(1, activation='sigmoid')
])

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

history = model.fit(x_train, y_train, epochs=3, verbose=2)

# Predict on the training data
predictions = model.predict(x_train)
print('Predictions:', predictions.flatten())

Important Notes

Bidirectional LSTM doubles the number of parameters because it runs two LSTMs.

Use return_sequences=True if you want to stack more recurrent layers.

Bidirectional LSTM works best when the entire sequence is available, not for streaming data.

Summary

Bidirectional LSTM reads sequences forward and backward to capture full context.

It improves performance on tasks like language understanding and time series analysis.

Use the Bidirectional wrapper around an LSTM layer in your model.