SimpleRNN layer in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output shape of SimpleRNN layer

What is the output shape of the following SimpleRNN layer when given input shape (batch_size=32, timesteps=10, features=8)?

TensorFlow

import tensorflow as tf

rnn = tf.keras.layers.SimpleRNN(16)
input_shape = (32, 10, 8)
inputs = tf.random.uniform(input_shape)
outputs = rnn(inputs)
print(outputs.shape)

A (32, 10, 16)

B (32, 16)

C (10, 16)

D (32, 8)

💡 Hint

SimpleRNN by default returns only the last output for each sequence.

The SimpleRNN layer with units=16 returns the last output for each sequence, so the output shape is (batch_size, units) = (32, 16).

❓ Model Choice

intermediate

Choosing SimpleRNN for sequence data

Which of the following tasks is best suited for a SimpleRNN layer?

A Predicting the next word in a sentence based on previous words

B Classifying images of cats and dogs

C Clustering customer data based on purchase history

D Reducing dimensionality of tabular data

💡 Hint

SimpleRNN is designed to process sequences over time.

SimpleRNN is a recurrent layer that processes sequences, making it suitable for tasks like predicting the next word in a sentence.

❓ Hyperparameter

advanced

Effect of return_sequences=True in SimpleRNN

What is the effect of setting return_sequences=True in a SimpleRNN layer?

A The layer returns the output at each timestep, producing a 3D tensor

B The layer returns the sum of all outputs across timesteps

C The layer returns the input sequence unchanged

D The layer returns only the last output, producing a 2D tensor

💡 Hint

Think about whether the output includes all timesteps or just the last one.

Setting return_sequences=True makes SimpleRNN return the output at every timestep, resulting in a 3D tensor (batch_size, timesteps, units).

❓ Metrics

advanced

Interpreting training loss for SimpleRNN model

A SimpleRNN model for sequence classification shows training loss decreasing steadily but validation loss increasing after some epochs. What does this indicate?

A The model is underfitting both training and validation data

B The model has a bug causing loss to increase

C The model is perfectly generalizing to new data

D The model is overfitting the training data

💡 Hint

Think about what it means when training loss improves but validation loss worsens.

When training loss decreases but validation loss increases, the model is learning training data too well and not generalizing, which is overfitting.

🔧 Debug

expert

Debugging SimpleRNN input shape error

Given this code snippet, what error will occur and why? import tensorflow as tf rnn = tf.keras.layers.SimpleRNN(10) inputs = tf.random.uniform((32, 10)) outputs = rnn(inputs)

TensorFlow

import tensorflow as tf
rnn = tf.keras.layers.SimpleRNN(10)
inputs = tf.random.uniform((32, 10))
outputs = rnn(inputs)

A No error, outputs shape will be (32, 10)

B TypeError because inputs are not integers

C ValueError because input must be 3D but got 2D tensor

D RuntimeError due to missing activation function

💡 Hint

Check the expected input shape for SimpleRNN layers.

SimpleRNN expects input shape (batch_size, timesteps, features). The input here is 2D (batch_size, timesteps) missing features dimension, causing ValueError.

Practice

1. What does the SimpleRNN layer in TensorFlow primarily do?

easy

A. Processes sequences step-by-step, remembering past information

B. Performs image classification using convolution

C. Reduces dimensionality of data using PCA

D. Generates random numbers for initialization

Solution

1. Step 1: Understand the role of SimpleRNN

   The SimpleRNN layer is designed to process sequential data by iterating through each time step and keeping track of previous information.

2. Step 2: Compare with other options

   Options A, B, and C describe unrelated tasks: random number generation (A), image processing (B), and dimensionality reduction (C), which are not the purpose of SimpleRNN.

3. Final Answer:

   Processes sequences step-by-step, remembering past information -> Option A

4. Quick Check:

   SimpleRNN = sequence processing [OK]

Hint: SimpleRNN remembers past steps in sequences [OK]

Common Mistakes:

• Confusing SimpleRNN with convolutional layers
• Thinking SimpleRNN generates random data
• Mixing SimpleRNN with dimensionality reduction methods

2. Which of the following is the correct way to add a SimpleRNN layer with 10 units in TensorFlow Keras?

easy

A. tf.layers.SimpleRNN(units=10)

B. tf.keras.SimpleRNNLayer(10)

C. tf.keras.layers.SimpleRNN(10)

D. tf.keras.layers.RNN(10)

Solution

1. Step 1: Recall correct TensorFlow Keras syntax

   The SimpleRNN layer is added using tf.keras.layers.SimpleRNN(units), where units is the number of neurons.

2. Step 2: Check other options for errors

   tf.keras.SimpleRNNLayer(10) uses a non-existent class name. tf.layers.SimpleRNN(units=10) uses outdated API. tf.keras.layers.RNN(10) uses the generic RNN class without specifying SimpleRNN.

3. Final Answer:

   tf.keras.layers.SimpleRNN(10) -> Option C

4. Quick Check:

   Correct class name and syntax = tf.keras.layers.SimpleRNN(10) [OK]

Hint: Use tf.keras.layers.SimpleRNN(units) to add layer [OK]

Common Mistakes:

• Using wrong class names like SimpleRNNLayer
• Confusing tf.layers with tf.keras.layers
• Using generic RNN without specifying SimpleRNN

3. What will be the output shape of the following model?

model = tf.keras.Sequential([
  tf.keras.layers.SimpleRNN(5, input_shape=(10, 3))
])
output_shape = model.output_shape

medium

A. (None, 5)

B. (None, 10, 5)

C. (10, 3, 5)

D. (None, 10, 3)

Solution

1. Step 1: Understand SimpleRNN output shape default

   By default, SimpleRNN returns only the last output for the sequence, so output shape is (batch_size, units). Here units=5.

2. Step 2: Analyze input shape and batch dimension

   Input shape is (10, 3) meaning 10 time steps and 3 features. Batch size is None (unspecified). So output shape is (None, 5).

3. Final Answer:

   (None, 5) -> Option A

4. Quick Check:

   Default return_sequences=false gives (None, units) [OK]

Hint: Default SimpleRNN outputs last step shape (None, units) [OK]

Common Mistakes:

• Assuming output shape includes time steps without return_sequences
• Confusing input shape with output shape
• Ignoring batch size dimension

4. Identify the error in this code snippet:

model = tf.keras.Sequential()
model.add(tf.keras.layers.SimpleRNN(8, input_shape=(5,)))
model.compile(optimizer='adam', loss='mse')

medium

A. SimpleRNN units must be a tuple, not an integer

B. Input shape should be 2D (timesteps, features), not (5,)

C. Loss function 'mse' is invalid for RNN

D. Optimizer 'adam' is not supported

Solution

1. Step 1: Check input_shape format for SimpleRNN

   SimpleRNN expects input_shape as (timesteps, features). Here input_shape=(5,) is missing the features dimension.

2. Step 2: Validate other parameters

   Units can be integer, 'mse' loss and 'adam' optimizer are valid choices for regression tasks.

3. Final Answer:

   Input shape should be 2D (timesteps, features), not (5,) -> Option B

4. Quick Check:

   SimpleRNN input_shape must be (timesteps, features) [OK]

Hint: Input shape for SimpleRNN must have two dimensions [OK]

Common Mistakes:

• Using 1D input shape instead of 2D
• Thinking units must be tuple
• Believing 'mse' loss is invalid for RNN

5. You want a SimpleRNN layer to output the full sequence of hidden states for each time step instead of just the last one. Which parameter should you set and how?

tf.keras.layers.SimpleRNN(4, input_shape=(7, 2), ???)

hard

A. unroll=true

B. return_state=true

C. stateful=true

D. return_sequences=true

Solution

1. Step 1: Understand output options of SimpleRNN

   By default, SimpleRNN returns only the last output. To get outputs for all time steps, set return_sequences=true.

2. Step 2: Check other parameters meanings

   return_state=true returns the last hidden state separately, stateful=true keeps states between batches, and unroll=true changes computation method but not output shape.

3. Final Answer:

   return_sequences=true -> Option D

4. Quick Check:

   Full sequence output = return_sequences=true [OK]

Hint: Set return_sequences=true for full sequence output [OK]

Common Mistakes:

• Confusing return_state with return_sequences
• Thinking stateful controls output shape
• Using unroll expecting output change