Bidirectional RNN in TensorFlow - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create a bidirectional RNN layer using TensorFlow.

TensorFlow

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.[1](
        tf.keras.layers.SimpleRNN(32), input_shape=(10, 8))
])

Drag options to blanks, or click blank then click option'

ABidirectional

BDense

CConv1D

DDropout

Attempts:

3 left

💡 Hint

Common Mistakes

Using Dense or Conv1D layers instead of Bidirectional for RNNs.

Not wrapping the RNN layer inside the Bidirectional wrapper.

The Bidirectional wrapper is used to create a bidirectional RNN layer in TensorFlow.

2fill in blank

medium

Complete the code to get the output shape of a bidirectional RNN layer.

TensorFlow

import tensorflow as tf

layer = tf.keras.layers.Bidirectional(tf.keras.layers.SimpleRNN(16))
input_data = tf.random.normal([5, 10, 8])
output = layer(input_data)
print(output.shape)  # Expected shape: (5, [1])

Drag options to blanks, or click blank then click option'

A16

B32

C8

D10

Attempts:

3 left

💡 Hint

Common Mistakes

Expecting output shape to be (5, 16) instead of (5, 32).

Confusing batch size with feature size.

The output size doubles because the RNN runs forward and backward, so 16 units become 32.

3fill in blank

hard

Fix the error in the code to correctly create a bidirectional LSTM layer.

TensorFlow

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Bidirectional(tf.keras.layers.[1](64), input_shape=(20, 10))
])

Drag options to blanks, or click blank then click option'

ALSTM

BDense

CSimpleRNN

DConv2D

Attempts:

3 left

💡 Hint

Common Mistakes

Using Dense or Conv2D layers inside Bidirectional.

Using SimpleRNN when LSTM is required.

The LSTM layer is the correct RNN type to use inside the Bidirectional wrapper for LSTM networks.

4fill in blank

hard

Fill both blanks to create a bidirectional GRU layer that returns sequences.

TensorFlow

import tensorflow as tf

layer = tf.keras.layers.Bidirectional(
    tf.keras.layers.[1](128, return_sequences=[2])
)

Drag options to blanks, or click blank then click option'

AGRU

BTrue

CFalse

DSimpleRNN

Attempts:

3 left

💡 Hint

Common Mistakes

Setting return_sequences to False when sequences are needed.

Using SimpleRNN instead of GRU.

Use GRU for the layer type and set return_sequences=True to get output at each time step.

5fill in blank

hard

Fill all three blanks to build a bidirectional RNN model with an LSTM layer, a dropout layer, and a dense output layer.

TensorFlow

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Bidirectional(tf.keras.layers.[1](64), input_shape=(15, 12)),
    tf.keras.layers.[2](0.5),
    tf.keras.layers.[3](10, activation='softmax')
])

Drag options to blanks, or click blank then click option'

ALSTM

BDropout

CDense

DSimpleRNN

Attempts:

3 left

💡 Hint

Common Mistakes

Using SimpleRNN instead of LSTM for the first layer.

Forgetting to add Dropout for regularization.

Using Dense without activation or wrong activation.

The model uses a LSTM inside Bidirectional, followed by a Dropout layer to reduce overfitting, and a Dense layer with softmax activation for classification output.

Practice

1. What is the main advantage of using a Bidirectional RNN compared to a standard RNN?

easy

A. It uses less memory by processing only half the sequence.

B. It trains faster because it skips some time steps.

C. It processes the input sequence in both forward and backward directions to capture more context.

D. It only works with fixed-length sequences.

Solution

1. Step 1: Understand how standard RNNs process sequences

   Standard RNNs read sequences only in one direction, usually forward, so they only capture past context.

2. Step 2: Recognize the bidirectional approach

   Bidirectional RNNs read sequences both forward and backward, allowing them to capture information from past and future context.

3. Final Answer:

   It processes the input sequence in both forward and backward directions to capture more context. -> Option C

4. Quick Check:

   Bidirectional RNN = forward + backward context [OK]

Hint: Bidirectional means reading sequence both ways [OK]

Common Mistakes:

• Thinking it reduces memory usage
• Assuming it skips time steps
• Believing it only works on fixed-length sequences

2. Which of the following is the correct way to create a Bidirectional LSTM layer in TensorFlow?

easy

A. tf.keras.layers.Bidirectional(units=64, layer=tf.keras.layers.LSTM())

B. tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(units=64))

C. tf.keras.layers.LSTM(units=64, bidirectional=True)

D. tf.keras.layers.LSTM(tf.keras.layers.Bidirectional(units=64))

Solution

1. Step 1: Recall the TensorFlow syntax for Bidirectional wrapper

   In TensorFlow, you wrap an RNN layer like LSTM inside tf.keras.layers.Bidirectional.

2. Step 2: Check each option

   tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(units=64)) correctly wraps LSTM with Bidirectional. Options A, B, and C misuse the syntax or parameters.

3. Final Answer:

   tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(units=64)) -> Option B

4. Quick Check:

   Bidirectional wraps RNN layer [OK]

Hint: Wrap RNN layer inside Bidirectional() [OK]

Common Mistakes:

• Passing bidirectional=True to LSTM directly
• Putting Bidirectional inside LSTM
• Incorrect parameter order

3. Consider the following TensorFlow code snippet:

import tensorflow as tf
model = tf.keras.Sequential([
  tf.keras.layers.Bidirectional(tf.keras.layers.SimpleRNN(4, return_sequences=True), input_shape=(5, 10)),
  tf.keras.layers.Dense(1)
])
output = model(tf.random.uniform((1, 5, 10)))
print(output.shape)

What will be the printed output shape?

medium

A. (1, 5, 1)

B. (1, 4, 1)

C. (1, 8, 1)

D. (5, 1)

Solution

1. Step 1: Analyze the Bidirectional SimpleRNN output shape

   SimpleRNN has 4 units, bidirectional doubles output units to 8. Since return_sequences=True, output shape is (batch, time_steps, 8).

2. Step 2: Check the Dense layer effect

   Dense(1) applies to last dimension, converting 8 features to 1, so output shape becomes (batch, time_steps, 1) = (1, 5, 1).

3. Final Answer:

   (1, 5, 1) -> Option A

4. Quick Check:

   Bidirectional doubles units, Dense changes features [OK]

Hint: Bidirectional doubles units; Dense changes last dim [OK]

Common Mistakes:

• Ignoring return_sequences=True effect
• Not doubling units for bidirectional
• Confusing batch and time dimensions

4. You wrote this code but get an error:

model = tf.keras.Sequential([
  tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32), input_shape=(None, 10)),
  tf.keras.layers.Dense(1)
])

The error says: ValueError: input_shape must be a tuple of integers, not NoneType
What is the likely cause and fix?

medium

A. Add return_sequences=True to the LSTM layer.

B. Remove the Bidirectional wrapper to fix the error.

C. Change LSTM units from 32 to 10.

D. Replace input_shape=(None, 10) with input_shape=(5, 10) or a fixed integer for time steps.

Solution

1. Step 1: Understand input_shape requirements

   TensorFlow expects input_shape to be a tuple of integers without None for time steps in Sequential models.

2. Step 2: Fix input_shape

   Replace input_shape=(None, 10) with a fixed integer like (5, 10) to specify sequence length.

3. Final Answer:

   Replace input_shape=(None, 10) with input_shape=(5, 10) or fixed integer -> Option D

4. Quick Check:

   input_shape must have fixed integers [OK]

Hint: Use fixed integers in input_shape, not None [OK]

Common Mistakes:

• Leaving None in input_shape for Sequential
• Removing Bidirectional unnecessarily
• Changing units instead of input shape

5. You want to build a sentiment analysis model using a Bidirectional GRU in TensorFlow. The input sequences have variable length. Which approach correctly handles variable-length sequences and avoids processing padded parts?

hard

A. Use tf.keras.layers.Bidirectional(tf.keras.layers.GRU(64)) with masking layer before it to ignore padded tokens.

B. Set return_sequences=False in GRU and ignore padding.

C. Manually trim all sequences to the same length before feeding to the model.

D. Use Bidirectional GRU without masking and rely on Dense layer to fix padding.

Solution

1. Step 1: Understand variable-length sequence handling

   Variable-length sequences are padded to same length; masking tells RNN to ignore padded parts.

2. Step 2: Use masking with Bidirectional GRU

   Adding a tf.keras.layers.Masking layer before Bidirectional GRU ensures padded tokens are ignored during processing.

3. Final Answer:

   Use Bidirectional GRU with masking layer before it -> Option A

4. Quick Check:

   Masking ignores padding in variable-length sequences [OK]

Hint: Add Masking layer before Bidirectional RNN for padding [OK]

Common Mistakes:

• Ignoring masking and expecting model to handle padding
• Trimming sequences manually losing data
• Relying on Dense layer to fix padding