RNN for text classification in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Model Choice

intermediate

Choosing the right RNN layer for text classification

You want to build a simple recurrent neural network for classifying movie reviews as positive or negative. Which RNN layer is best suited to capture the sequence information while keeping the model simple?

A Use a Conv1D layer because convolution is better for sequence data than RNNs.

B Use a Dense layer directly on the input sequences without any recurrent layers.

C Use a MaxPooling layer to reduce sequence length before classification.

D Use a SimpleRNN layer because it is the most basic recurrent layer and captures sequence order.

💡 Hint

Think about which layer type is designed to handle sequences step-by-step.

SimpleRNN is a recurrent layer designed to process sequences step-by-step, capturing order and dependencies. Conv1D and MaxPooling are not recurrent and Dense layers alone do not handle sequence order.

❓ Predict Output

intermediate

Output shape of RNN layer in text classification model

Consider the following Keras model snippet for text classification:

model = Sequential()
model.add(Embedding(input_dim=10000, output_dim=64, input_length=100))
model.add(SimpleRNN(32))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

print(model.layers[1].output_shape)

What will be printed as the output shape?

A (None, 32)

B (None, 100, 32)

C (None, 64)

D (None, 1)

💡 Hint

SimpleRNN returns the last output by default, not the full sequence.

The SimpleRNN layer with 32 units outputs a vector of size 32 for each input sample. The batch size is None (unspecified), so the output shape is (None, 32).

❓ Hyperparameter

advanced

Choosing the right sequence length for RNN input

You are training an RNN for text classification on movie reviews. Your dataset has reviews of varying lengths, but you must fix the input length for the model. What is the best approach to set the sequence length?

A Set the sequence length to a fixed value like 100, truncating longer reviews and padding shorter ones.

B Set the sequence length to the maximum review length in the dataset to keep all information.

C Set the sequence length to the average review length without padding or truncation.

D Do not fix sequence length; feed variable-length sequences directly to the RNN.

💡 Hint

Think about model input requirements and efficiency.

RNN models require fixed input lengths. Setting a fixed length like 100 and padding/truncating sequences is standard practice to balance information retention and computational efficiency.

❓ Metrics

advanced

Interpreting training metrics for RNN text classifier

After training an RNN text classification model, you observe the following metrics on the validation set:

• Loss: 0.65
• Accuracy: 0.60

What does this tell you about the model's performance?

A The model is performing well with high accuracy and low loss.

B The model has perfect predictions on the validation set.

C The model is underfitting and not learning enough from the data.

D The model is overfitting the training data.

💡 Hint

Consider what accuracy and loss values mean for classification.

An accuracy of 60% and loss of 0.65 on validation is low performance, indicating the model is underfitting and not capturing patterns well.

🔧 Debug

expert

Debugging vanishing gradient in RNN training

You train a SimpleRNN model for text classification but notice the training loss barely decreases after many epochs. Which of the following is the most likely cause?

A The dataset is too large, causing slow training.

B The model suffers from vanishing gradients due to the SimpleRNN architecture.

C The optimizer learning rate is too high, causing divergence.

D The batch size is too small, causing unstable updates.

💡 Hint

Think about common problems with SimpleRNN layers in deep sequence models.

SimpleRNN layers often suffer from vanishing gradients, making it hard to learn long-range dependencies, causing training loss to stagnate.