GRU for text in NLP

Introduction

GRU helps computers understand text by remembering important words and forgetting less important ones. It makes reading and predicting text easier and faster.

When you want to predict the next word in a sentence.

When you need to classify text like spam or not spam.

When you want to understand the sentiment of a review.

When you want to translate text from one language to another.

When you want to summarize a long article into a short one.

Syntax

NLP

torch.nn.GRU(input_size, hidden_size, num_layers=1, batch_first=False, dropout=0, bidirectional=False)

input_size is the number of features in each input word vector.

hidden_size is how many features the GRU remembers at each step.

Examples

Creates a GRU that takes 10 features per word and remembers 20 features.

NLP

gru = torch.nn.GRU(input_size=10, hidden_size=20)

Creates a 2-layer GRU that expects batches first and remembers 100 features.

NLP

gru = torch.nn.GRU(input_size=50, hidden_size=100, num_layers=2, batch_first=True)

Sample Model

This code creates a small GRU to process two short sentences. Each word is a vector of 5 numbers. The GRU remembers 4 features at each step. We print the output and hidden states shapes and values.

NLP

import torch
import torch.nn as nn

# Sample text data: batch of 2 sentences, each with 3 words, each word represented by 5 features
input_data = torch.randn(2, 3, 5)  # batch_size=2, seq_len=3, input_size=5

# Create GRU: input_size=5, hidden_size=4, batch_first=True
gru = nn.GRU(input_size=5, hidden_size=4, batch_first=True)

# Forward pass
output, hidden = gru(input_data)

print('Output shape:', output.shape)
print('Output:', output)
print('Hidden shape:', hidden.shape)
print('Hidden:', hidden)

Important Notes

GRU is faster and simpler than LSTM but still remembers important information.

Set batch_first=True if your input shape is (batch, sequence, features).

Hidden state shape is (num_layers * num_directions, batch, hidden_size).

Summary

GRU helps models remember important parts of text while ignoring less important parts.

Use GRU for tasks like text prediction, classification, and translation.

Input shape and hidden size must match your data and task needs.