nn.LSTM layer in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output shape of nn.LSTM with batch_first=True

Consider the following PyTorch code snippet using nn.LSTM with batch_first=True. What is the shape of the output tensor `out`?

PyTorch

import torch
import torch.nn as nn

lstm = nn.LSTM(input_size=10, hidden_size=20, num_layers=2, batch_first=True)
input_tensor = torch.randn(5, 7, 10)  # batch=5, seq_len=7, input_size=10
out, (h_n, c_n) = lstm(input_tensor)
print(out.shape)

A torch.Size([7, 5, 20])

B torch.Size([5, 20, 7])

C torch.Size([7, 20, 5])

D torch.Size([5, 7, 20])

💡 Hint

Remember that batch_first=True means the batch dimension is the first dimension in the input and output.

With batch_first=True, the input and output tensors have shape (batch_size, seq_len, hidden_size). Here batch_size=5, seq_len=7, hidden_size=20, so output shape is (5, 7, 20).

❓ Model Choice

intermediate

Choosing LSTM parameters for sequence classification

You want to build an LSTM model to classify sequences of length 15 with 8 features each into 3 classes. Which nn.LSTM configuration is correct to output a tensor suitable for classification after processing the entire sequence?

A nn.LSTM(input_size=3, hidden_size=8, num_layers=2, batch_first=True)

B nn.LSTM(input_size=15, hidden_size=3, num_layers=1, batch_first=False)

C nn.LSTM(input_size=8, hidden_size=16, num_layers=1, batch_first=True)

D nn.LSTM(input_size=8, hidden_size=15, num_layers=3, batch_first=False)

💡 Hint

Input size should match the feature dimension per time step.

The input_size must be 8 because each time step has 8 features. Batch_first=True is convenient for batch processing. Hidden size 16 is a reasonable choice. Other options mismatch input_size or batch_first setting.

❓ Hyperparameter

advanced

Effect of increasing num_layers in nn.LSTM

What is the main effect of increasing the num_layers parameter in nn.LSTM from 1 to 3?

A The LSTM will have 3 stacked layers, allowing it to learn more complex temporal patterns.

B The input size of the LSTM will automatically triple.

C The output tensor shape will change from 3D to 2D.

D The LSTM will process sequences in reverse order.

💡 Hint

Think about what stacking layers means in neural networks.

Increasing num_layers stacks multiple LSTM layers on top of each other, enabling the model to capture more complex features over time. It does not change input size or output tensor dimensions directly.

🔧 Debug

advanced

Identifying error in LSTM input shape

What error will this code raise when running the LSTM forward pass?

PyTorch

import torch
import torch.nn as nn

lstm = nn.LSTM(input_size=5, hidden_size=10, batch_first=True)
input_tensor = torch.randn(4, 6, 4)  # batch=4, seq_len=6, input_size=4
out, (h_n, c_n) = lstm(input_tensor)

A RuntimeError: input size mismatch. Expected input_size=5 but got 4.

B TypeError: LSTM input must be 2D tensor.

C ValueError: batch_first must be False for this input shape.

D No error, code runs successfully.

💡 Hint

Check the input tensor's last dimension against the LSTM's input_size.

The LSTM expects input tensors with last dimension equal to input_size=5, but input_tensor has last dimension 4, causing a RuntimeError.

❓ Metrics

expert

Interpreting LSTM hidden state shapes after forward pass

After running an nn.LSTM with num_layers=2, hidden_size=8, batch_first=True on input of shape (3, 10, 6), what is the shape of the hidden state tensor h_n?

A torch.Size([10, 3, 8])

B torch.Size([2, 3, 8])

C torch.Size([3, 10, 8])

D torch.Size([3, 2, 8])

💡 Hint

Hidden state shape is (num_layers, batch_size, hidden_size).

The hidden state h_n has shape (num_layers, batch_size, hidden_size). Here num_layers=2, batch_size=3, hidden_size=8, so shape is (2, 3, 8).