SimpleRNN layer in TensorFlow

Introduction

The SimpleRNN layer helps a model remember information from earlier in a sequence. It is useful for tasks where order and time matter, like understanding sentences or time series.

When you want to predict the next word in a sentence based on previous words.

When analyzing time-based data like stock prices or weather changes.

When building a simple chatbot that needs to remember recent messages.

When processing sequences like sensor readings over time.

When learning basic recurrent neural networks before moving to more complex ones.

Syntax

TensorFlow

tf.keras.layers.SimpleRNN(units, activation='tanh', return_sequences=False, return_state=False, go_backwards=False, stateful=False, dropout=0.0, recurrent_dropout=0.0)

units is the number of memory cells in the layer.

return_sequences=True makes the layer output the full sequence, not just the last output.

Examples

A SimpleRNN layer with 32 memory units, outputs only the last output by default.

TensorFlow

SimpleRNN(32)

Uses ReLU activation and returns the full sequence of outputs.

TensorFlow

SimpleRNN(64, activation='relu', return_sequences=True)

Processes the input sequence backwards.

TensorFlow

SimpleRNN(10, go_backwards=True)

Sample Model

This code creates random sequence data and passes it through a SimpleRNN layer with 4 units. It prints the input and output shapes and the actual output values.

TensorFlow

import numpy as np
import tensorflow as tf

# Create sample data: batch of 2 sequences, each with 5 time steps and 3 features
x = np.random.random((2, 5, 3)).astype(np.float32)

# Build model with SimpleRNN layer
model = tf.keras.Sequential([
    tf.keras.layers.SimpleRNN(4, activation='tanh', return_sequences=False, input_shape=(5, 3))
])

# Compile model
model.compile(optimizer='adam', loss='mse')

# Run a forward pass to get output
output = model(x)

print('Input shape:', x.shape)
print('Output shape:', output.shape)
print('Output values:', output.numpy())

Important Notes

SimpleRNN is good for learning basic sequence patterns but can struggle with long sequences due to forgetting.

For longer sequences, consider using LSTM or GRU layers which remember better.

Always set input_shape in the first layer to tell the model the shape of your data.

Summary

SimpleRNN layer processes sequences step-by-step, remembering past information.

It outputs either the last step or the full sequence depending on return_sequences.

Good for simple sequence tasks but limited for long-term memory.