nn.GRU layer in PyTorch - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does the nn.GRU layer in PyTorch do?

The nn.GRU layer processes sequences by using Gated Recurrent Units to keep track of information over time, helping models understand order and context in data like sentences or time series.

intermediate

What are the main components inside a GRU cell?

A GRU cell has two gates: the update gate, which decides how much past information to keep, and the reset gate, which decides how to combine new input with past memory.

beginner

How do you create a simple nn.GRU layer in PyTorch for input size 10 and hidden size 20?

Use: nn.GRU(input_size=10, hidden_size=20). This sets the input feature size to 10 and the hidden state size to 20.

intermediate

What is the shape of the output from nn.GRU when batch_first=True and input shape is (batch, seq_len, input_size)?

The output shape is (batch, seq_len, num_directions * hidden_size). It gives the hidden states for each time step in the sequence.

intermediate

Why might you choose GRU over LSTM in a model?

GRUs are simpler and faster to train because they have fewer gates than LSTMs, but still handle sequence data well, making them good for smaller datasets or faster experiments.

What does the update gate in a GRU control?

A How much past information to keep

B How to reset the hidden state

C The input feature size

D The output sequence length

The update gate controls how much of the past information is kept in the current hidden state.

In PyTorch, what argument makes nn.GRU expect input shape as (batch, seq_len, input_size)?

A dropout=0.5

B bidirectional=True

C num_layers=2

D batch_first=True

Setting batch_first=True makes the input and output tensors have batch size as the first dimension.

Which of these is NOT a gate in a GRU cell?

A Forget gate

B Update gate

C Reset gate

D None of the above

Forget gate is part of LSTM, not GRU.

What is the main advantage of GRU compared to LSTM?

A Requires more memory

B Handles longer sequences better

C Simpler and faster to train

D Has more gates

GRUs have fewer gates, making them simpler and faster to train.

What does the hidden_size parameter in nn.GRU specify?

A The length of the input sequence

B The size of the hidden state vector

C The number of layers

D The batch size

hidden_size sets the dimension of the hidden state vector in the GRU.

Explain how a GRU layer processes sequence data and why it is useful.

Think about how GRU keeps important information from the past while reading new data.

Describe how to set up and use an nn.GRU layer in PyTorch including input and output shapes.

Consider the shape of input and output tensors and the key parameters.