Bidirectional RNNs in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Bidirectional RNN Outputs

In a bidirectional RNN, what does the output at each time step represent?

A The output is the sum of all hidden states across all time steps.

B The output only contains information from past time steps up to the current step.

C The output only contains information from future time steps after the current step.

D The output combines information from past and future time steps relative to the current step.

💡 Hint

Think about how information flows in both directions in a bidirectional RNN.

Bidirectional RNNs process sequences forward and backward, so the output at each time step includes context from both past and future relative to that step.

❓ Predict Output

intermediate

Output Shape of a Bidirectional RNN in PyTorch

Given the following PyTorch code, what is the shape of output?

PyTorch

import torch
import torch.nn as nn

rnn = nn.RNN(input_size=10, hidden_size=20, num_layers=1, bidirectional=True)
input_seq = torch.randn(5, 3, 10)  # seq_len=5, batch=3, input_size=10
output, hidden = rnn(input_seq)
print(output.shape)

A torch.Size([3, 5, 40])

B torch.Size([5, 3, 20])

C torch.Size([5, 3, 40])

D torch.Size([3, 5, 20])

💡 Hint

Remember the output shape is (seq_len, batch, num_directions * hidden_size).

Since the RNN is bidirectional, the output hidden size doubles (20 * 2 = 40). The output shape is (5, 3, 40).

❓ Model Choice

advanced

Choosing Bidirectional RNN for Sequence Tasks

For which of the following tasks is using a bidirectional RNN most beneficial?

A Classifying the sentiment of a complete sentence after reading it entirely.

B Predicting the next word in a sentence given only previous words.

C Generating text one character at a time from left to right.

D Real-time speech recognition where future audio frames are not available.

💡 Hint

Consider if future context is available when making predictions.

Bidirectional RNNs use both past and future context, so they work best when the entire sequence is available, like sentence classification.

❓ Hyperparameter

advanced

Effect of Bidirectionality on Hidden State Size

If you set hidden_size=50 in a bidirectional RNN with num_layers=1, what is the size of the hidden state tensor h_n returned by PyTorch?

A torch.Size([2, batch_size, 50])

B torch.Size([1, batch_size, 100])

C torch.Size([1, batch_size, 50])

D torch.Size([2, batch_size, 100])

💡 Hint

Remember the first dimension of h_n is num_layers * num_directions.

With 1 layer and 2 directions, the first dimension is 2. Hidden size per direction is 50, so shape is (2, batch_size, 50).

🔧 Debug

expert

Identifying Error in Bidirectional RNN Output Usage

Consider this PyTorch code snippet:

import torch
import torch.nn as nn

rnn = nn.RNN(input_size=8, hidden_size=16, bidirectional=True)
input_seq = torch.randn(7, 4, 8)
output, hidden = rnn(input_seq)
last_output = output[-1]
print(last_output.shape)

What is the shape of last_output and why might using it directly be problematic for sequence classification?

A torch.Size([4, 16]); It only contains forward direction output, missing backward context.

B torch.Size([4, 32]); It combines forward and backward outputs at the last time step, but backward output corresponds to the first input step, so it may not represent the full sequence context.

C torch.Size([7, 4, 32]); It is the full output sequence, so no problem using it directly.

D torch.Size([4, 16]); It only contains backward direction output, missing forward context.

💡 Hint

Think about what the last time step output means in a bidirectional RNN.

The output at the last time step concatenates forward output at last step and backward output at first step. Using it directly may not capture full sequence meaning for classification.