Bidirectional LSTM in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output shape of Bidirectional LSTM layer

Consider the following Keras code snippet that creates a Bidirectional LSTM layer. What is the shape of the output tensor after passing an input batch of shape (32, 10, 8) through this layer?

from tensorflow.keras.layers import Bidirectional, LSTM
from tensorflow.keras.models import Sequential

model = Sequential()
model.add(Bidirectional(LSTM(16, return_sequences=False), input_shape=(10, 8)))

output = model.layers[0].output_shape

NLP

from tensorflow.keras.layers import Bidirectional, LSTM
from tensorflow.keras.models import Sequential

model = Sequential()
model.add(Bidirectional(LSTM(16, return_sequences=False), input_shape=(10, 8)))

output = model.layers[0].output_shape

A (None, 16)

B (None, 32)

C (None, 64)

D (None, 10, 32)

💡 Hint

Remember that Bidirectional doubles the units of the LSTM output when return_sequences=False.

The LSTM has 16 units. Bidirectional wraps it and concatenates forward and backward outputs, doubling the units to 32. Since return_sequences=False, output shape is (batch_size, 32). Batch size is None (unspecified).

❓ Model Choice

intermediate

Choosing Bidirectional LSTM for sequence classification

You want to build a model to classify movie reviews as positive or negative based on the text. Which model architecture below best uses a Bidirectional LSTM for this task?

A Embedding -> Dense(64, activation='relu') -> Bidirectional LSTM (units=64) -> Dense(1, activation='sigmoid')

B Embedding -> LSTM (units=64, return_sequences=True) -> Dense(1, activation='sigmoid')

C Embedding -> Bidirectional LSTM (units=64, return_sequences=True) -> Dense(1, activation='sigmoid')

D Embedding -> Bidirectional LSTM (units=64, return_sequences=False) -> Dense(1, activation='sigmoid')

💡 Hint

For classification, the LSTM should output a single vector per sample, not a sequence.

Option D uses Bidirectional LSTM with return_sequences=False, producing one vector per input sequence, suitable for classification. Options A and C output sequences, which is not ideal here. Option D incorrectly places a Dense layer before LSTM, which is not standard.

❓ Hyperparameter

advanced

Effect of return_sequences in Bidirectional LSTM

What is the effect of setting return_sequences=True in a Bidirectional LSTM layer in Keras?

A The layer returns the input sequence unchanged.

B The layer returns the cell state instead of the hidden state.

C The layer returns the hidden state for each time step, producing a 3D tensor output.

D The layer returns only the last hidden state as a 2D tensor output.

💡 Hint

Think about whether the output keeps the time dimension or not.

When return_sequences=True, the LSTM returns the hidden state for every time step, so the output shape is (batch_size, time_steps, units*2) for Bidirectional. If False, it returns only the last hidden state (batch_size, units*2).

❓ Metrics

advanced

Interpreting training metrics of Bidirectional LSTM

You train a Bidirectional LSTM model for sentiment analysis. After 10 epochs, training accuracy is 95% but validation accuracy is 60%. What does this indicate?

A The model is overfitting the training data and not generalizing well.

B The model is underfitting and needs more training epochs.

C The model has a bug causing incorrect validation evaluation.

D The validation data is easier than the training data.

💡 Hint

High training accuracy but low validation accuracy usually means the model memorizes training data.

High training accuracy with low validation accuracy means the model learned the training data too well but fails to generalize to new data, a classic sign of overfitting.

🔧 Debug

expert

Debugging shape mismatch in Bidirectional LSTM model

You have this Keras model code snippet:

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, Bidirectional, LSTM, Dense

model = Sequential()
model.add(Embedding(input_dim=1000, output_dim=64, input_length=20))
model.add(Bidirectional(LSTM(32)))
model.add(Dense(10, activation='softmax'))

model.compile(optimizer='adam', loss='categorical_crossentropy')

# You try to train with labels shape (batch_size, 20, 10) but get a shape error.

What is the cause of the shape error?

NLP

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, Bidirectional, LSTM, Dense

model = Sequential()
model.add(Embedding(input_dim=1000, output_dim=64, input_length=20))
model.add(Bidirectional(LSTM(32)))
model.add(Dense(10, activation='softmax'))

model.compile(optimizer='adam', loss='categorical_crossentropy')

# Training labels shape: (batch_size, 20, 10)

A The model outputs shape (batch_size, 10) but labels have shape (batch_size, 20, 10), causing mismatch.

B The Embedding layer output shape is incompatible with LSTM input shape.

C The Dense layer activation 'softmax' is invalid for multi-class classification.

D The loss function 'categorical_crossentropy' requires labels to be integers, not one-hot encoded.

💡 Hint

Check the output shape of the Bidirectional LSTM and the shape of the labels.

The Bidirectional LSTM without return_sequences outputs (batch_size, 64). The Dense layer outputs (batch_size, 10). But the labels have shape (batch_size, 20, 10), implying a sequence of labels per time step. This mismatch causes the error.