nn.RNN layer in PyTorch - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the purpose of the nn.RNN layer in PyTorch?

The nn.RNN layer processes sequences of data by passing information from one time step to the next, allowing the model to learn patterns over time.

beginner

What are the main inputs to an nn.RNN layer?

The main inputs are the sequence data (shape: seq_len, batch_size, input_size) and an optional initial hidden state (shape: num_layers * num_directions, batch_size, hidden_size).

beginner

What does the 'hidden_size' parameter control in nn.RNN?

It controls the size of the hidden state vector, which stores information from previous time steps and affects the model's capacity to learn patterns.

intermediate

How does nn.RNN handle multiple layers and directions?

You can set 'num_layers' to stack multiple RNN layers, and 'bidirectional=True' to process sequences forwards and backwards, doubling the output size.

beginner

What are the outputs of nn.RNN layer?

It outputs 'output' (all hidden states for each time step) and 'hidden' (the last hidden state for each layer and direction).

What shape should the input sequence to nn.RNN have?

A batch_size, seq_len, input_size

B seq_len, batch_size, input_size

C input_size, seq_len, batch_size

D batch_size, input_size, seq_len

The nn.RNN expects input with shape (seq_len, batch_size, input_size).

What does setting 'bidirectional=True' do in nn.RNN?

A Processes the sequence forwards and backwards

B Stacks multiple RNN layers

C Changes the activation function

D Disables the hidden state

Bidirectional RNN processes the input sequence in both forward and backward directions.

Which of these is NOT an output of nn.RNN?

A Output for all time steps

B Last hidden state

C Predicted class labels

D None of the above

nn.RNN outputs hidden states, not direct class predictions.

What does the 'hidden_size' parameter affect?

A Size of the hidden state vector

B Length of the input sequence

C Number of layers

D Batch size

'hidden_size' controls the size of the hidden state vector in the RNN.

How can you provide an initial hidden state to nn.RNN?

A By setting a parameter during initialization

B You cannot provide an initial hidden state

C By modifying the input sequence

D By passing it as the second argument to the forward method

You can pass the initial hidden state as the second argument when calling the RNN layer.

Explain how the nn.RNN layer processes a sequence of data step-by-step.

Think about how information flows through time steps in the RNN.

Describe the difference between a unidirectional and bidirectional nn.RNN layer.

Consider how reading the sequence backwards adds information.