nn.RNN layer in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output shape of nn.RNN with batch_first=True

Consider the following PyTorch code using nn.RNN with batch_first=True. What is the shape of the output tensor `out`?

PyTorch

import torch
import torch.nn as nn

rnn = nn.RNN(input_size=5, hidden_size=3, num_layers=1, batch_first=True)
input_tensor = torch.randn(4, 7, 5)  # batch=4, seq_len=7, input_size=5
out, hn = rnn(input_tensor)
print(out.shape)

A torch.Size([4, 7, 3])

B torch.Size([7, 4, 3])

C torch.Size([4, 3, 7])

D torch.Size([7, 3, 4])

💡 Hint

Remember that batch_first=True means the batch dimension is the first dimension in the input and output.

With batch_first=True, the input and output tensors have shape (batch_size, seq_len, hidden_size). Here, batch_size=4, seq_len=7, hidden_size=3, so output shape is (4, 7, 3).

❓ Model Choice

intermediate

Choosing RNN parameters for sequence classification

You want to build a simple RNN model to classify sequences of length 10 with 8 features each into 4 classes. Which nn.RNN configuration is correct for this task?

A nn.RNN(input_size=8, hidden_size=4, num_layers=1, batch_first=False)

B nn.RNN(input_size=10, hidden_size=4, num_layers=1, batch_first=False)

C nn.RNN(input_size=4, hidden_size=10, num_layers=2, batch_first=True)

D nn.RNN(input_size=8, hidden_size=16, num_layers=1, batch_first=True)

💡 Hint

Input size should match the number of features per time step.

The input_size must be 8 because each time step has 8 features. batch_first=True is common for batch processing. hidden_size can be any positive integer; 16 is reasonable. num_layers=1 is fine for a simple model.

❓ Hyperparameter

advanced

Effect of increasing num_layers in nn.RNN

What is the main effect of increasing the num_layers parameter in nn.RNN from 1 to 3?

A The input size of the RNN will automatically triple.

B The RNN will have 3 stacked recurrent layers, allowing it to learn more complex patterns.

C The output size of the RNN will be three times larger.

D The RNN will process sequences three times faster.

💡 Hint

Think about what stacking layers means in neural networks.

Increasing num_layers stacks multiple RNN layers. Each layer processes the output of the previous, enabling the model to learn more complex temporal features. It does not change input size or speed directly.

🔧 Debug

advanced

Identifying error in nn.RNN input shape

What error will this code raise when running the RNN forward pass?

PyTorch

import torch
import torch.nn as nn

rnn = nn.RNN(input_size=6, hidden_size=4, num_layers=1)
input_tensor = torch.randn(5, 6, 6)  # batch=5, seq_len=6, input_size=6
out, hn = rnn(input_tensor)

A RuntimeError: Expected input of shape (seq_len, batch, input_size), but got (batch, seq_len, input_size)

B No error, runs successfully

C TypeError: input_tensor must be a list, not a tensor

D ValueError: hidden_size must be equal to input_size

💡 Hint

Check the default expected input shape for nn.RNN when batch_first is False.

By default, nn.RNN expects input shape (seq_len, batch, input_size). The input tensor here has shape (batch, seq_len, input_size), causing a RuntimeError.

❓ Metrics

expert

Interpreting training loss behavior of nn.RNN model

You train an nn.RNN model for sequence prediction. The training loss decreases steadily, but the validation loss starts increasing after some epochs. What does this indicate?

A The learning rate is too low, causing slow convergence.

B The model is underfitting and needs more training.

C The model is overfitting the training data and not generalizing well.

D The batch size is too large, causing unstable training.

💡 Hint

Think about what it means when validation loss increases while training loss decreases.

When training loss decreases but validation loss increases, the model fits training data too closely and fails to generalize, which is called overfitting.