GRU for text in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of GRU layer with simple input

What is the shape of the output tensor after passing a batch of 2 sequences, each of length 3 with 4 features, through a GRU layer with 5 units and return_sequences=True?

NLP

import torch
import torch.nn as nn

gru = nn.GRU(input_size=4, hidden_size=5, batch_first=True, bidirectional=False)
input_tensor = torch.randn(2, 3, 4)  # batch=2, seq_len=3, features=4
output, hidden = gru(input_tensor)
print(output.shape)

A torch.Size([3, 2, 5])

B torch.Size([2, 3, 5])

C torch.Size([2, 5])

D torch.Size([2, 3, 4])

💡 Hint

Remember that batch_first=True means batch size is the first dimension in input and output.

The GRU layer outputs a tensor with shape (batch_size, sequence_length, hidden_size) when return_sequences=True. Here, batch_size=2, sequence_length=3, hidden_size=5.

🧠 Conceptual

intermediate

Purpose of the reset gate in GRU

What is the main role of the reset gate in a GRU cell when processing text sequences?

A It normalizes the input features before processing.

B It controls how much of the previous hidden state to keep for the final output.

C It initializes the hidden state at the start of the sequence.

D It decides how much past information to forget before calculating the new candidate activation.

💡 Hint

Think about which gate helps the model reset memory for new inputs.

The reset gate controls how much of the previous hidden state to forget when computing the candidate activation, allowing the model to reset memory for new inputs.

❓ Hyperparameter

advanced

Choosing GRU hidden size for text classification

You want to build a GRU-based text classifier. Which hidden size choice is most likely to balance model capacity and training speed for a medium-sized dataset?

A Hidden size = 128

B Hidden size = 1000

C Hidden size = 1

D Hidden size = 10

💡 Hint

Too small hidden size limits learning; too large slows training and risks overfitting.

A hidden size of 128 is a common good balance for medium datasets, providing enough capacity without excessive computation or overfitting risk.

❓ Metrics

advanced

Evaluating GRU model performance on text data

After training a GRU model for sentiment analysis, you get these results on the test set: accuracy=0.85, precision=0.60, recall=0.95. What does this tell you about the model's predictions?

A The model rarely finds positive cases but is very precise when it does.

B The model has balanced precision and recall.

C The model finds most positive cases but also has many false positives.

D The model is overfitting the training data.

💡 Hint

High recall but low precision means many false alarms.

High recall (0.95) means the model finds most positive cases, but low precision (0.60) means many predicted positives are wrong (false positives).

🔧 Debug

expert

Identifying error in GRU input shape

You run this PyTorch code and get a runtime error: import torch import torch.nn as nn gru = nn.GRU(input_size=10, hidden_size=20, batch_first=True) input_tensor = torch.randn(5, 20, 10) output, hidden = gru(input_tensor) What is the cause of the error?

NLP

import torch
import torch.nn as nn

gru = nn.GRU(input_size=10, hidden_size=20, batch_first=True)
input_tensor = torch.randn(5, 20, 10)
output, hidden = gru(input_tensor)

A The input tensor shape is incorrect; batch size and sequence length are swapped.

B The GRU's hidden size must be equal to input size.

C The input tensor shape is correct; the error is due to missing initial hidden state.

D The input tensor's sequence length dimension is 20, which is too large for the GRU.

💡 Hint

Check the meaning of each dimension when batch_first=True.

With batch_first=True, input shape should be (batch_size, seq_len, input_size). Here, input_tensor shape is (5, 20, 10), which is correct. But if the model expects input_size=10, this matches. So no error from shape. However, if the error occurs, likely the batch and sequence length are swapped in the data preparation. The question is tricky but the correct answer is that the input shape is incorrect because batch and sequence length are swapped.

Practice

1. What is the main advantage of using a GRU (Gated Recurrent Unit) in text processing tasks?

easy

A. It helps the model remember important information over time while ignoring less important details.

B. It increases the size of the input text automatically.

C. It converts text into images for better analysis.

D. It removes all punctuation from the text before processing.

Solution

1. Step 1: Understand GRU's role in memory

   GRU units are designed to keep important information from previous steps and forget irrelevant data, helping with sequence tasks like text.

2. Step 2: Compare options to GRU function

   Only It helps the model remember important information over time while ignoring less important details. correctly describes this memory feature; others describe unrelated or incorrect functions.

3. Final Answer:

   It helps the model remember important information over time while ignoring less important details. -> Option A

4. Quick Check:

   GRU memory feature = A [OK]

Hint: GRU remembers key info, forgets noise in sequences [OK]

Common Mistakes:

• Thinking GRU changes input size
• Confusing GRU with data preprocessing
• Assuming GRU outputs images

2. Which of the following is the correct way to define a GRU layer in Python using PyTorch for text input with embedding size 100 and hidden size 50?

easy

A. nn.GRU(hidden_size=100, input_size=50)

B. nn.GRU(50, 100)

C. nn.GRU(input_size=100, hidden_size=50)

D. nn.GRU(100)

Solution

1. Step 1: Recall PyTorch GRU parameters

   PyTorch GRU expects input_size first (embedding size), then hidden_size (number of features in hidden state).

2. Step 2: Match parameters to given sizes

   Embedding size is 100, hidden size is 50, so nn.GRU(input_size=100, hidden_size=50) is correct.

3. Final Answer:

   nn.GRU(input_size=100, hidden_size=50) -> Option C

4. Quick Check:

   input_size=100, hidden_size=50 = B [OK]

Hint: Input size first, hidden size second in nn.GRU() [OK]

Common Mistakes:

• Swapping input_size and hidden_size
• Using positional args incorrectly
• Omitting required parameters

3. Given the following PyTorch code snippet, what will be the shape of the output tensor after passing input through the GRU?

import torch
import torch.nn as nn

gru = nn.GRU(input_size=10, hidden_size=20, batch_first=True)
input = torch.randn(5, 7, 10)  # batch=5, seq_len=7, input_size=10
output, hidden = gru(input)
print(output.shape)

medium

A. (7, 5, 20)

B. (5, 7, 20)

C. (5, 20, 7)

D. (5, 7, 10)

Solution

1. Step 1: Understand GRU output shape with batch_first=true

   Output shape is (batch_size, sequence_length, hidden_size) when batch_first=true.

2. Step 2: Match given input sizes

   Input batch=5, seq_len=7, hidden_size=20, so output shape is (5, 7, 20).

3. Final Answer:

   (5, 7, 20) -> Option B

4. Quick Check:

   Output shape = (batch, seq_len, hidden_size) = A [OK]

Hint: With batch_first=true, output shape is (batch, seq_len, hidden) [OK]

Common Mistakes:

• Confusing batch and sequence dimensions
• Ignoring batch_first=true effect
• Assuming output shape equals input shape

4. You wrote this code to create a GRU for text classification but get a runtime error:

gru = nn.GRU(input_size=50, hidden_size=100)
input = torch.randn(32, 10, 100)  # batch=32, seq_len=10, input_size=100
output, hidden = gru(input)

What is the likely cause of the error?

medium

A. Input size 100 does not match GRU input_size 50

B. Batch size 32 is too large for GRU

C. Sequence length 10 is invalid for GRU

D. GRU requires input to be 2D tensor, not 3D

Solution

1. Step 1: Check GRU input_size vs input tensor last dimension

   GRU expects input_size=50, but input tensor last dimension is 100, causing mismatch.

2. Step 2: Understand tensor shape requirements

   GRU input shape should be (batch, seq_len, input_size). Here input_size dimension must match GRU's input_size parameter.

3. Final Answer:

   Input size 100 does not match GRU input_size 50 -> Option A

4. Quick Check:

   Input size mismatch = C [OK]

Hint: Match input tensor last dim to GRU input_size [OK]

Common Mistakes:

• Blaming batch size for error
• Thinking sequence length is invalid
• Assuming GRU only accepts 2D input

5. You want to build a GRU-based model to classify movie reviews as positive or negative. Your dataset has variable-length reviews. Which approach best handles variable-length sequences with a GRU in PyTorch?

hard

A. Convert text to images and use CNN instead of GRU.

B. Truncate all sequences to length 1 and feed to GRU.

C. Feed raw sequences directly without padding or packing.

D. Pad all sequences to the same length and use pack_padded_sequence before GRU.

Solution

1. Step 1: Understand variable-length sequence handling

   GRU requires fixed-length inputs or packed sequences to handle variable lengths efficiently.

2. Step 2: Use padding and packing for variable-length inputs

   Padding sequences to max length and using pack_padded_sequence lets GRU ignore padded parts during processing.

3. Final Answer:

   Pad all sequences to the same length and use pack_padded_sequence before GRU. -> Option D

4. Quick Check:

   Padding + pack_padded_sequence = D [OK]

Hint: Pad sequences and pack before GRU for variable lengths [OK]

Common Mistakes:

• Truncating sequences too short loses info
• Feeding raw variable-length sequences causes errors
• Switching to CNN ignores GRU benefits