nn.GRU layer in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output shape of nn.GRU layer

Given the following PyTorch code, what is the shape of the output tensor out after running the GRU layer?

PyTorch

import torch
import torch.nn as nn

gru = nn.GRU(input_size=10, hidden_size=20, num_layers=2)
input_tensor = torch.randn(5, 3, 10)  # (seq_len, batch, input_size)
out, h_n = gru(input_tensor)
print(out.shape)

A torch.Size([3, 20, 5])

B torch.Size([3, 5, 20])

C torch.Size([5, 20, 3])

D torch.Size([5, 3, 20])

💡 Hint

Remember the GRU output shape is (seq_len, batch, hidden_size).

The GRU layer outputs a tensor with shape (sequence length, batch size, hidden size). Here, seq_len=5, batch=3, hidden_size=20.

❓ Model Choice

intermediate

Choosing GRU for sequence data

You want to build a model to predict the next word in a sentence using sequential text data. Which of the following is the best reason to choose an nn.GRU layer over a simple RNN layer?

A GRU can better capture long-term dependencies with fewer parameters than a simple RNN.

B GRU layers are faster to train because they use convolution operations.

C Simple RNNs have built-in attention mechanisms, but GRUs do not.

D GRU layers require the input to be one-hot encoded, unlike simple RNNs.

💡 Hint

Think about how GRUs handle memory compared to simple RNNs.

GRUs use gating mechanisms to better remember long-term information and avoid vanishing gradients, making them more effective than simple RNNs for sequence tasks.

❓ Hyperparameter

advanced

Effect of num_layers in nn.GRU

What is the effect of increasing the num_layers parameter in an nn.GRU layer from 1 to 3?

A The GRU will have 3 stacked layers, allowing it to learn more complex features from the sequence.

B The GRU will process the input sequence 3 times in parallel and average the results.

C The GRU will increase the hidden size by 3 times automatically.

D The GRU will reduce the sequence length by a factor of 3.

💡 Hint

Think about what stacking layers means in neural networks.

Increasing num_layers stacks multiple GRU layers, where each layer's output is the next layer's input, enabling the model to learn hierarchical features.

❓ Metrics

advanced

Interpreting GRU training loss

During training of a GRU-based model for time series prediction, the training loss decreases steadily but the validation loss starts increasing after some epochs. What does this indicate?

A The GRU layer is not suitable for time series data.

B The model is underfitting and needs more training epochs.

C The model is overfitting the training data and not generalizing well.

D The learning rate is too low, causing slow convergence.

💡 Hint

Think about what it means when validation loss rises but training loss falls.

When training loss decreases but validation loss increases, the model memorizes training data but fails to generalize, a sign of overfitting.

🔧 Debug

expert

Identifying error in GRU input shape

You run this code and get a runtime error. What is the cause?

import torch
import torch.nn as nn

gru = nn.GRU(input_size=8, hidden_size=16)
input_tensor = torch.randn(4, 10)  # Missing batch dimension
out, h_n = gru(input_tensor)

A Hidden size must be equal to input size, but here they differ.

B Input tensor must be 3D with shape (seq_len, batch, input_size), but input_tensor is 2D.

C GRU requires input tensor to be on GPU but input_tensor is on CPU.

D The batch size must be the first dimension, but here it is the second.

💡 Hint

Check the expected input shape for nn.GRU layers.

nn.GRU expects input tensors of shape (sequence length, batch size, input size). The given input is missing the batch dimension, causing a runtime error.