nn.GRU layer in PyTorch - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create a GRU layer with input size 10 and hidden size 20.

PyTorch

gru = nn.GRU(input_size=[1], hidden_size=20)

Drag options to blanks, or click blank then click option'

A10

B20

C5

D15

Attempts:

3 left

💡 Hint

Common Mistakes

Confusing input_size with hidden_size.

Using hidden_size value for input_size.

The input_size parameter defines the number of expected features in the input. Here, it should be 10.

2fill in blank

medium

Complete the code to pass an input tensor of shape (5, 3, 10) through the GRU layer.

PyTorch

output, hidden = gru([1])

Drag options to blanks, or click blank then click option'

Adata

Binput

Cinput_tensor

Dx

Attempts:

3 left

💡 Hint

Common Mistakes

Using undefined variable names.

Passing the wrong tensor shape.

The variable holding the input tensor is named input_tensor, which matches the expected input shape.

3fill in blank

hard

Fix the error in the code by completing the missing argument to initialize the hidden state for batch size 3 and hidden size 20.

PyTorch

hidden = torch.zeros(1, [1], 20)

Drag options to blanks, or click blank then click option'

A1

B3

C20

D5

Attempts:

3 left

💡 Hint

Common Mistakes

Using hidden size instead of batch size.

Using number of layers instead of batch size.

The hidden state shape is (num_layers * num_directions, batch_size, hidden_size). Here batch_size is 3.

4fill in blank

hard

Fill both blanks to create a GRU layer with 2 layers and batch_first=True.

PyTorch

gru = nn.GRU(input_size=10, hidden_size=20, num_layers=[1], batch_first=[2])

Drag options to blanks, or click blank then click option'

A2

BTrue

CFalse

D1

Attempts:

3 left

💡 Hint

Common Mistakes

Confusing num_layers with hidden_size.

Setting batch_first to False by mistake.

num_layers=2 sets two stacked GRU layers. batch_first=True makes input shape (batch, seq, feature).

5fill in blank

hard

Fill both blanks to extract the last output from the GRU output tensor of shape (5, 3, 20).

PyTorch

last_output = output[[1], [2], :]

Drag options to blanks, or click blank then click option'

A4

B2

C:

D0

Attempts:

3 left

💡 Hint

Common Mistakes

Using wrong sequence index.

Using wrong batch index.

Not selecting all features.

The last output is at sequence index 4 (0-based), batch index 0, and all features (:).

Practice

1. What is the primary purpose of the nn.GRU layer in PyTorch?

easy

A. To reduce the dimensionality of data using PCA

B. To perform image classification using convolution

C. To process sequential data by remembering past information

D. To generate random numbers for initialization

Solution

1. Step 1: Understand the role of GRU

   The GRU (Gated Recurrent Unit) is designed to handle sequences by keeping track of past inputs, which helps in tasks like text or speech processing.

2. Step 2: Compare with other options

   The other options describe unrelated tasks: dimensionality reduction using PCA, image classification using convolution, and random number generation, which are not the purpose of GRU.

3. Final Answer:

   To process sequential data by remembering past information -> Option C

4. Quick Check:

   GRU = sequence memory [OK]

Hint: GRU remembers past inputs in sequences [OK]

Common Mistakes:

• Confusing GRU with convolution layers
• Thinking GRU reduces data dimensions like PCA
• Assuming GRU generates random values

2. Which of the following is the correct way to create a GRU layer with input size 10 and hidden size 20 in PyTorch?

easy

A. nn.GRU(20, 10)

B. nn.GRU(input_size=10, hidden_size=20)

C. nn.GRU(hidden_size=10, input_size=20)

D. nn.GRU(10)

Solution

1. Step 1: Recall GRU constructor parameters

   The correct order and names are input_size first, then hidden_size. So nn.GRU(input_size=10, hidden_size=20) is correct.

2. Step 2: Check other options

   nn.GRU(20, 10) reverses the sizes. nn.GRU(hidden_size=10, input_size=20) swaps parameter names incorrectly. nn.GRU(10) misses the hidden size parameter.

3. Final Answer:

   nn.GRU(input_size=10, hidden_size=20) -> Option B

4. Quick Check:

   Input size first, hidden size second [OK]

Hint: Remember: input_size before hidden_size in nn.GRU [OK]

Common Mistakes:

• Swapping input_size and hidden_size
• Omitting hidden_size parameter
• Using wrong parameter names

3. Given the following code, what is the shape of the output tensor out?

import torch
import torch.nn as nn

gru = nn.GRU(input_size=5, hidden_size=3, batch_first=True)
x = torch.randn(4, 7, 5)  # batch=4, seq_len=7, input_size=5
out, h_n = gru(x)
print(out.shape)

medium

A. (4, 7, 3)

B. (7, 4, 3)

C. (4, 3, 7)

D. (7, 3, 4)

Solution

1. Step 1: Understand batch_first=True effect

   With batch_first=True, input shape is (batch, seq_len, input_size). Output shape matches (batch, seq_len, hidden_size).

2. Step 2: Apply shapes from code

   Input is (4, 7, 5), hidden_size=3, so output out shape is (4, 7, 3).

3. Final Answer:

   (4, 7, 3) -> Option A

4. Quick Check:

   Output shape = (batch, seq_len, hidden_size) [OK]

Hint: batch_first=True means batch is first dimension [OK]

Common Mistakes:

• Confusing batch and sequence dimensions
• Ignoring batch_first parameter
• Mixing hidden_size with input_size

4. Which of the following correctly describes the execution of this code snippet?

import torch
import torch.nn as nn

gru = nn.GRU(input_size=8, hidden_size=4)
x = torch.randn(5, 10, 8)
out, h = gru(x)
print(out.shape)

medium

A. The code runs without errors and prints (5, 10, 4)

B. The hidden_size must be larger than input_size

C. The GRU layer requires batch_first=True for this input shape

D. The input tensor shape is incorrect for default GRU settings

Solution

1. Step 1: Check default GRU input expectations

   By default, GRU expects input shape (seq_len, batch, input_size). Here, input is (5, 10, 8), so seq_len=5, batch=10, input_size=8 which matches default.

2. Step 2: Verify output shape

   Output shape will be (seq_len, batch, hidden_size) = (5, 10, 4).

3. Step 3: Evaluate statements

   The code runs without errors and prints (5, 10, 4). Hidden_size can be smaller than input_size. batch_first=True is not required. Input shape is correct for default settings.

4. Final Answer:

   The code runs without errors and prints (5, 10, 4) -> Option A

5. Quick Check:

   Default GRU input shape = (seq_len, batch, input_size) [OK]

Hint: Default GRU expects seq_len first, batch second [OK]

Common Mistakes:

• Assuming batch is first dimension without batch_first=True
• Thinking hidden_size must be bigger than input_size
• Expecting output shape to swap batch and seq_len

5. You want to build a GRU-based model to process variable-length sequences in a batch. Which approach correctly handles this in PyTorch?

hard

A. Feed raw variable-length sequences directly to nn.GRU without padding

B. Manually truncate all sequences to the shortest length before input

C. Use nn.GRU with batch_first=False and ignore sequence lengths

D. Pad sequences to the same length and use pack_padded_sequence before feeding to nn.GRU

Solution

1. Step 1: Understand variable-length sequence handling

   PyTorch requires sequences in a batch to be the same length or packed. Padding sequences and using pack_padded_sequence allows GRU to ignore padded parts.

2. Step 2: Evaluate options

   Pad sequences to the same length and use pack_padded_sequence before feeding to nn.GRU correctly pads and packs sequences. Feed raw variable-length sequences directly to nn.GRU without padding is invalid because GRU cannot handle raw variable-length sequences. Use nn.GRU with batch_first=False and ignore sequence lengths ignores lengths, causing wrong results. Manually truncate all sequences to the shortest length before input loses data by truncation.

3. Final Answer:

   Pad sequences and use pack_padded_sequence before nn.GRU -> Option D

4. Quick Check:

   Use padding + packing for variable-length sequences [OK]

Hint: Pad then pack sequences before GRU [OK]

Common Mistakes:

• Feeding variable-length sequences without padding
• Ignoring sequence lengths in batch
• Truncating sequences losing data