Bird
0
0

Why might an RNN-based text generation model struggle with very long sequences, and what is a common solution?

hard📝 Conceptual Q10 of 15
NLP - Text Generation
Why might an RNN-based text generation model struggle with very long sequences, and what is a common solution?
ARNNs cannot process sequences longer than 10 tokens; use CNNs instead
BRNNs overfit on long sequences; reduce training data size
CRNNs forget long-term dependencies; use LSTM or GRU cells instead
DRNNs require one-hot encoding for long sequences; switch to embeddings
Step-by-Step Solution
Solution:

  1. Step 1: Identify RNN limitation

    Standard RNNs have difficulty remembering information over long sequences due to vanishing gradients.

  2. Step 2: Recognize common solution

    LSTM and GRU cells are designed to keep long-term dependencies better, solving this problem.

  3. Final Answer:

    RNNs forget long-term dependencies; use LSTM or GRU cells instead -> Option C

  4. Quick Check:

    Use LSTM/GRU to handle long-term dependencies [OK]
Quick Trick: Use LSTM/GRU to remember long sequences [OK]
Common Mistakes:
MISTAKES
  • Thinking RNNs can't process sequences longer than 10
  • Confusing overfitting with forgetting
  • Believing one-hot encoding solves long-term memory

Want More Practice?

15+ quiz questions · All difficulty levels · Free

Free Signup - Practice All Questions
More NLP Quizzes
Sentiment Analysis Advanced - Multilingual sentiment - Quiz 4medium Sequence Models for NLP - Why sequence models understand word order - Quiz 5medium Sequence Models for NLP - Why sequence models understand word order - Quiz 13medium Sequence Models for NLP - Attention mechanism basics - Quiz 3easy Text Similarity and Search - Edit distance (Levenshtein) - Quiz 4medium Text Similarity and Search - Semantic similarity with embeddings - Quiz 6medium Topic Modeling - LDA with scikit-learn - Quiz 12easy Topic Modeling - Choosing number of topics - Quiz 1easy Word Embeddings - Training Word2Vec with Gensim - Quiz 2easy Word Embeddings - FastText embeddings - Quiz 13medium