LSTM for text in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of LSTM layer with return_sequences=True

What is the shape of the output tensor after passing input of shape (32, 10, 50) through an LSTM layer with 64 units and return_sequences=True?

NLP

import tensorflow as tf
input_tensor = tf.random.uniform((32, 10, 50))
lstm_layer = tf.keras.layers.LSTM(64, return_sequences=True)
output = lstm_layer(input_tensor)
print(output.shape)

A (10, 64)

B (32, 64)

C (32, 10, 64)

D (32, 10, 50)

💡 Hint

Remember that return_sequences=True returns output for each time step.

When return_sequences=True, the LSTM returns output for every time step, so the output shape is (batch_size, time_steps, units). Here, batch_size=32, time_steps=10, units=64.

❓ Model Choice

intermediate

Choosing LSTM for text classification

You want to classify movie reviews as positive or negative using their text. Which model architecture is best suited for this task?

A A simple feedforward neural network with bag-of-words input

B An LSTM network processing word sequences

C A convolutional neural network on raw text characters

D A linear regression model on word counts

💡 Hint

Think about models that capture word order and context.

LSTM networks are designed to process sequences and capture context over time, making them well suited for text classification tasks where word order matters.

❓ Hyperparameter

advanced

Effect of increasing LSTM units

What is the most likely effect of increasing the number of units in an LSTM layer from 50 to 200 when training on a small text dataset?

A The model may overfit and training time will increase

B The model will ignore word order

C The model will train faster and generalize better

D The model will underfit due to fewer parameters

💡 Hint

More units mean more parameters to learn.

Increasing units increases model capacity and parameters, which can cause overfitting on small datasets and increase training time.

❓ Metrics

advanced

Evaluating LSTM text model with imbalanced classes

You trained an LSTM model for spam detection on a dataset where 90% of messages are not spam. Which metric is best to evaluate your model's performance?

A Accuracy

B Confusion matrix only

C Mean Squared Error

D Precision and Recall

💡 Hint

Think about metrics that handle class imbalance well.

Accuracy can be misleading with imbalanced data. Precision and recall give better insight into how well the model detects the minority class (spam).

🔧 Debug

expert

Why does this LSTM model not learn?

You trained an LSTM model on text data but the training loss does not decrease. The model code is below. What is the most likely cause?

NLP

import tensorflow as tf
model = tf.keras.Sequential([
    tf.keras.layers.Embedding(input_dim=10000, output_dim=64, input_length=100),
    tf.keras.layers.LSTM(128),
    tf.keras.layers.Dense(1, activation='sigmoid')
])
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
# Training data: X_train shape (1000, 100), y_train shape (1000,)
model.fit(X_train, y_train, epochs=10, batch_size=32)

A The input sequences are not padded to the same length

B The Embedding layer output_dim is too small

C The model lacks an activation function in the LSTM layer

D The loss function is incorrect for binary classification

💡 Hint

Check the input data shape and preprocessing.

LSTM layers require input sequences to be the same length. If sequences are not padded, the model input shape is inconsistent, causing training issues.