LSTM layer in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output shape of LSTM layer with return_sequences

Consider the following TensorFlow code snippet creating an LSTM layer:

import tensorflow as tf

lstm_layer = tf.keras.layers.LSTM(10, return_sequences=True)
input_tensor = tf.random.uniform((32, 5, 8))  # batch=32, time_steps=5, features=8
output = lstm_layer(input_tensor)
print(output.shape)

What is the printed output shape?

TensorFlow

import tensorflow as tf

lstm_layer = tf.keras.layers.LSTM(10, return_sequences=True)
input_tensor = tf.random.uniform((32, 5, 8))
output = lstm_layer(input_tensor)
print(output.shape)

A (5, 32, 10)

B (32, 10)

C (32, 5, 10)

D (32, 8, 10)

💡 Hint

Remember that return_sequences=True makes the LSTM return output for each time step.

The input shape is (batch_size=32, time_steps=5, features=8). With return_sequences=True, the output shape is (batch_size, time_steps, units), so (32, 5, 10).

❓ Model Choice

intermediate

Choosing LSTM for sequence data

You want to build a model to predict the next word in a sentence based on previous words. Which model layer is best suited for this task?

A Convolutional layer

B LSTM layer

C Dense layer with ReLU activation

D Dropout layer

💡 Hint

Think about which layer can remember information over time steps.

LSTM layers are designed to handle sequence data and remember information over time, making them ideal for predicting the next word in a sentence.

❓ Hyperparameter

advanced

Effect of increasing LSTM units

What is the most likely effect of increasing the number of units in an LSTM layer from 50 to 200?

A Model capacity increases, possibly improving accuracy but increasing risk of overfitting

B Model will train faster due to fewer parameters

C Model will ignore earlier time steps

D Model will require less memory

💡 Hint

More units mean more parameters and complexity.

Increasing units increases model capacity and parameters, which can improve accuracy but also increase overfitting risk and training time.

❓ Metrics

advanced

Interpreting LSTM training loss curve

During training an LSTM model, you observe the training loss steadily decreases but the validation loss starts increasing after some epochs. What does this indicate?

A There is a bug in the loss calculation

B The model is underfitting the training data

C The model has converged perfectly

D The model is overfitting the training data

💡 Hint

Think about what it means when validation loss worsens but training loss improves.

When training loss decreases but validation loss increases, the model is learning the training data too well and not generalizing, which is overfitting.

🔧 Debug

expert

Identifying error in LSTM input shape

Given this code snippet:

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.LSTM(32, input_shape=(10,)),
    tf.keras.layers.Dense(1)
])

input_data = tf.random.uniform((64, 10, 5))
output = model(input_data)
print(output.shape)

What error will this code raise?

A ValueError: Input 0 of layer 'lstm' is incompatible with the layer: expected shape=(None, 10), found shape=(64, 10, 5)

B TypeError: Cannot convert input data to tensor

C RuntimeError: LSTM layer requires 3D input but got 2D

D No error, output shape is (64, 1)

💡 Hint

Check the input_shape parameter and the actual input data shape.

The LSTM layer expects input shape (time_steps, features). Here, input_shape=(10,) means time_steps=10 but features dimension is missing. The input data has shape (64, 10, 5), so features=5. This mismatch causes a ValueError.