Text generation with RNN in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Model Choice

intermediate

Choosing the right RNN layer for text generation

You want to build a text generation model using TensorFlow. Which RNN layer is best suited to capture long-range dependencies in text?

A tf.keras.layers.LSTM

B tf.keras.layers.SimpleRNN

C tf.keras.layers.Dense

D tf.keras.layers.Conv1D

💡 Hint

Think about which layer can remember information over many time steps.

LSTM layers are designed to remember information for long sequences, making them ideal for text generation tasks.

❓ Predict Output

intermediate

Output shape of RNN layer in text generation model

Consider this TensorFlow code snippet for a text generation model:

model = tf.keras.Sequential([
  tf.keras.layers.Embedding(input_dim=1000, output_dim=64),
  tf.keras.layers.LSTM(128, return_sequences=True),
  tf.keras.layers.Dense(1000)
])

output_shape = model.output_shape

What is the value of output_shape?

A (None, None, 128)

B (None, 128)

C (None, 64)

D (None, None, 1000)

💡 Hint

Check the last Dense layer's output units and the return_sequences=True setting.

The LSTM outputs a sequence (because return_sequences=True), so the shape is (batch_size, sequence_length, 128). The Dense layer maps each time step to 1000 units, so final output shape is (batch_size, sequence_length, 1000).

❓ Hyperparameter

advanced

Effect of sequence length on training RNN for text generation

When training an RNN for text generation, what is the main effect of increasing the input sequence length?

A It reduces the model's ability to learn long-term dependencies.

B It increases the computational cost and memory usage during training.

C It decreases the vocabulary size the model can handle.

D It makes the model ignore the order of words in the sequence.

💡 Hint

Think about how longer sequences affect the amount of data processed per training step.

Longer input sequences mean the model processes more time steps per batch, increasing computation and memory needs.

❓ Metrics

advanced

Choosing the right metric for text generation model evaluation

Which metric is most appropriate to evaluate the quality of a text generation RNN model during training?

A F1 Score

B Mean Squared Error (MSE)

C Perplexity

D Accuracy

💡 Hint

Consider a metric that measures how well the model predicts the next word probabilities.

Perplexity measures how well a probability model predicts a sample. Lower perplexity means better predictions in language models.

🔧 Debug

expert

Debugging vanishing gradient in RNN text generation model

You trained an RNN text generation model but it fails to learn long-term dependencies. Which of the following is the most likely cause?

A Using SimpleRNN layers without gating mechanisms causing vanishing gradients

B Setting the learning rate too high causing unstable training

C Using too large batch size causing overfitting

D Using LSTM layers instead of SimpleRNN layers

💡 Hint

Think about which RNN type struggles with remembering information over many steps.

SimpleRNN layers lack gating mechanisms and often suffer from vanishing gradients, making it hard to learn long-term dependencies.