LSTM layer in TensorFlow - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to add an LSTM layer with 50 units to the model.

TensorFlow

model.add(tf.keras.layers.LSTM([1]))

Drag options to blanks, or click blank then click option'

Aactivation='relu'

BDense

C50

Dinput_shape=(10, 1)

Attempts:

3 left

💡 Hint

Common Mistakes

Using layer types like 'Dense' instead of a number.

Passing activation functions inside LSTM constructor incorrectly.

The LSTM layer requires the number of units as an integer. Here, 50 units are specified.

2fill in blank

medium

Complete the code to define the input shape for the LSTM layer with 100 time steps and 20 features.

TensorFlow

model.add(tf.keras.layers.LSTM(64, input_shape=[1]))

Drag options to blanks, or click blank then click option'

A(100, 20)

B[100, 20]

C(20, 100)

D100, 20

Attempts:

3 left

💡 Hint

Common Mistakes

Swapping time steps and features order.

Using list instead of tuple for input_shape.

The input_shape for LSTM is a tuple (timesteps, features). Here, 100 time steps and 20 features are correct.

3fill in blank

hard

Fix the error in the code by completing the LSTM layer to return sequences for stacking.

TensorFlow

model.add(tf.keras.layers.LSTM(32, return_sequences=[1]))

Drag options to blanks, or click blank then click option'

ATrue

BNone

C0

DFalse

Attempts:

3 left

💡 Hint

Common Mistakes

Leaving return_sequences as False causes shape errors when stacking.

Using None or 0 instead of True.

Setting return_sequences=True makes the LSTM output a sequence, which is needed when stacking LSTM layers.

4fill in blank

hard

Fill both blanks to create a stacked LSTM model with 64 units in the first layer returning sequences, and 32 units in the second layer.

TensorFlow

model = tf.keras.Sequential()
model.add(tf.keras.layers.LSTM([1], return_sequences=[2], input_shape=(50, 10)))
model.add(tf.keras.layers.LSTM(32))

Drag options to blanks, or click blank then click option'

A64

BTrue

CFalse

D32

Attempts:

3 left

💡 Hint

Common Mistakes

Setting return_sequences to False in the first layer.

Using wrong number of units in the first layer.

The first LSTM layer has 64 units and must return sequences (True) to feed the next LSTM layer.

5fill in blank

hard

Fill all three blanks to build a model with an LSTM layer of 100 units, dropout rate 0.2, and a Dense output layer with 1 unit.

TensorFlow

model = tf.keras.Sequential([
    tf.keras.layers.LSTM([1], dropout=[2], input_shape=(30, 5)),
    tf.keras.layers.Dense([3])
])

Drag options to blanks, or click blank then click option'

A100

B0.2

C1

D0.5

Attempts:

3 left

💡 Hint

Common Mistakes

Using dropout values outside 0 to 1 range.

Setting wrong number of units in Dense layer.

The LSTM layer has 100 units with dropout 0.2, and the Dense layer outputs 1 unit for prediction.

Practice

1. What is the main purpose of an LSTM layer in a neural network?

easy

A. To remember important information from past sequence data

B. To perform image classification

C. To reduce the size of input data

D. To generate random numbers

Solution

1. Step 1: Understand LSTM function

   LSTM layers are designed to handle sequence data by remembering important past information, unlike regular layers.

2. Step 2: Compare options

   Options A, B, and C describe unrelated tasks. Only To remember important information from past sequence data correctly describes LSTM's purpose.

3. Final Answer:

   To remember important information from past sequence data -> Option A

4. Quick Check:

   LSTM purpose = Remember past info [OK]

Hint: LSTM = memory for sequences, not general tasks [OK]

Common Mistakes:

• Confusing LSTM with convolutional layers
• Thinking LSTM reduces data size
• Assuming LSTM generates random outputs

2. Which of the following is the correct way to add an LSTM layer with 50 units using the `units` keyword in TensorFlow Keras?

easy

A. tf.keras.layers.LSTM(50)

B. tf.keras.layers.LSTM(units=50)

C. tf.keras.layers.LSTM(50, return_sequences=True)

D. tf.keras.layers.LSTM(50, dropout=0.2)

Solution

1. Step 1: Recall LSTM syntax

   The LSTM layer requires the number of units as a named argument 'units=50' for clarity and correctness.

2. Step 2: Evaluate options

   tf.keras.layers.LSTM(50) is allowed but less clear. Options C and D add extra parameters but the question asks for the correct basic syntax using the 'units' keyword.

3. Final Answer:

   tf.keras.layers.LSTM(units=50) -> Option B

4. Quick Check:

   LSTM syntax = units=number [OK]

Hint: Use named argument 'units' for clarity in LSTM layer [OK]

Common Mistakes:

• Omitting 'units=' keyword
• Confusing dropout with units parameter
• Adding unnecessary parameters when not asked

3. What will be the output shape of the following LSTM layer if the input shape is (batch_size, 10, 8)?

tf.keras.layers.LSTM(16, return_sequences=False)

medium

A. (batch_size, 8)

B. (batch_size, 10, 16)

C. (batch_size, 16)

D. (batch_size, 10, 8)

Solution

1. Step 1: Understand return_sequences effect

   When return_sequences=False, the LSTM returns only the last output for the sequence, so output shape is (batch_size, units).

2. Step 2: Apply to given parameters

   Units=16, so output shape is (batch_size, 16). Options B and D include sequence length, which is incorrect here.

3. Final Answer:

   (batch_size, 16) -> Option C

4. Quick Check:

   return_sequences=False output = (batch_size, units) [OK]

Hint: return_sequences=False means output shape excludes time steps [OK]

Common Mistakes:

• Assuming output keeps sequence length when return_sequences=False
• Confusing input shape with output shape
• Mixing units and input feature dimensions

4. Identify the error in this LSTM layer code snippet:

model = tf.keras.Sequential()
model.add(tf.keras.layers.LSTM(32, input_shape=(20, 5), return_sequences=False))
model.add(tf.keras.layers.LSTM(16))
model.add(tf.keras.layers.Dense(1))

medium

A. The first LSTM layer must have return_sequences=True

B. The input_shape should be (5, 20) instead of (20, 5)

C. The second LSTM layer should have return_sequences=True

D. Dense layer cannot follow LSTM layers

Solution

1. Step 1: Check LSTM stacking rules

   When stacking LSTM layers, all except the last must have return_sequences=True to pass full sequence outputs to the next layer.

2. Step 2: Analyze given code

   The first LSTM has return_sequences=False, outputting only (batch_size, 32). The second LSTM expects 3D sequence input but receives 2D, causing a shape incompatibility error.

3. Final Answer:

   The first LSTM layer must have return_sequences=True -> Option A

4. Quick Check:

   Stacked LSTM earlier layers need return_sequences=True [OK]

Hint: Only last LSTM can have return_sequences=False [OK]

Common Mistakes:

• Forgetting return_sequences=True in middle LSTM layers
• Mixing input_shape dimensions
• Thinking Dense can't follow LSTM

5. You want to build an LSTM model that outputs a sequence of the same length as input (10 time steps) with 20 features each. Which configuration achieves this?

hard

A. tf.keras.layers.LSTM(20, return_sequences=False, input_shape=(10, 20))

B. tf.keras.layers.LSTM(20, return_sequences=True, input_shape=(20, 10))

C. tf.keras.layers.LSTM(10, return_sequences=True, input_shape=(20, 20))

D. tf.keras.layers.LSTM(20, return_sequences=True, input_shape=(10, 20))

Solution

1. Step 1: Understand input and output shapes

   Input shape is (10, 20) meaning 10 time steps, 20 features. To output sequence of same length, return_sequences must be True.

2. Step 2: Check options

   tf.keras.layers.LSTM(20, return_sequences=True, input_shape=(10, 20)) matches input shape and sets return_sequences=True with units=20, so output shape is (batch_size, 10, 20). Options A, C and D have wrong return_sequences or input shapes.

3. Final Answer:

   tf.keras.layers.LSTM(20, return_sequences=True, input_shape=(10, 20)) -> Option D

4. Quick Check:

   return_sequences=True + correct input shape = desired output [OK]

Hint: Use return_sequences=True to keep sequence length [OK]

Common Mistakes:

• Setting return_sequences=False loses sequence length
• Mixing time steps and feature dimensions in input_shape
• Confusing units with input feature size