nn.LSTM layer in PyTorch - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does the nn.LSTM layer in PyTorch do?

The nn.LSTM layer processes sequences of data by remembering information over time. It helps models learn patterns in sequences like sentences or time series.

beginner

What are the main inputs and outputs of an nn.LSTM layer?

Input: a sequence of data with shape (sequence_length, batch_size, input_size). Output: the hidden states for each time step and the final hidden and cell states.

intermediate

Why does nn.LSTM have both hidden state and cell state?

The hidden state carries short-term memory, while the cell state carries long-term memory. This helps the LSTM remember important information over many steps.

beginner

How do you initialize an nn.LSTM layer for input size 10 and hidden size 20?

Use nn.LSTM(input_size=10, hidden_size=20). This sets the input feature size to 10 and the hidden layer size to 20.

intermediate

What does setting batch_first=True do in nn.LSTM?

It changes the input and output shape to (batch_size, sequence_length, input_size), which can be easier to work with when batches come first.

What shape does nn.LSTM expect for its input by default?

A (input_size, sequence_length, batch_size)

B (batch_size, input_size, sequence_length)

C (batch_size, sequence_length, input_size)

D (sequence_length, batch_size, input_size)

By default, nn.LSTM expects input shaped as (sequence_length, batch_size, input_size).

What are the two states returned by nn.LSTM besides the output?

A hidden state and cell state

B input state and output state

C weight state and bias state

D activation state and dropout state

nn.LSTM returns hidden state and cell state to keep track of short-term and long-term memory.

What does the hidden_size parameter control in nn.LSTM?

A The batch size

B The number of features in the hidden state

C The length of the input sequence

D The number of layers

hidden_size sets how many features the hidden state will have.

If batch_first=True, what is the input shape for nn.LSTM?

A (batch_size, input_size, sequence_length)

B (sequence_length, batch_size, input_size)

C (batch_size, sequence_length, input_size)

D (input_size, batch_size, sequence_length)

Setting batch_first=True changes input shape to (batch_size, sequence_length, input_size).

Why is nn.LSTM better than a simple RNN for long sequences?

A Because it can remember information longer using cell state

B Because it uses convolution layers

C Because it has fewer parameters

D Because it does not use activation functions

nn.LSTM uses a cell state to keep long-term memory, helping with long sequences.

Explain how nn.LSTM processes a sequence of data step-by-step.

Think about how information flows through time steps and how memory is kept.

Describe the difference between hidden state and cell state in nn.LSTM and why both are important.

Consider how remembering recent vs. older information helps understanding sequences.