Transformer decoder in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output shape of Transformer decoder layer

Given the following PyTorch code snippet for a Transformer decoder layer, what is the shape of the output tensor?

PyTorch

import torch
import torch.nn as nn

decoder_layer = nn.TransformerDecoderLayer(d_model=512, nhead=8)
memory = torch.rand(10, 32, 512)  # (sequence_length, batch_size, d_model)
tgt = torch.rand(20, 32, 512)     # (target_sequence_length, batch_size, d_model)
output = decoder_layer(tgt, memory)
print(output.shape)

A torch.Size([20, 32, 512])

B torch.Size([10, 32, 512])

C torch.Size([32, 20, 512])

D torch.Size([20, 512, 32])

💡 Hint

Remember the output shape matches the target input shape for the decoder layer.

The Transformer decoder layer outputs a tensor with the same shape as the target input: (target_sequence_length, batch_size, d_model). Here, tgt has shape (20, 32, 512), so output shape is (20, 32, 512).

🧠 Conceptual

intermediate

Purpose of the memory input in Transformer decoder

In a Transformer decoder, what is the role of the 'memory' input?

A It stores the previous decoder outputs for autoregressive generation.

B It provides the encoded information from the encoder to guide decoding.

C It contains the positional encodings for the target sequence.

D It is used to compute the loss during training.

💡 Hint

Think about how the decoder uses information from the encoder.

The 'memory' input is the output of the encoder. The decoder attends to this memory to incorporate encoded source information when generating outputs.

❓ Hyperparameter

advanced

Effect of increasing nhead in Transformer decoder

What is the effect of increasing the 'nhead' parameter in a Transformer decoder layer?

A It increases the batch size during training.

B It increases the size of the feedforward network inside the decoder layer.

C It increases the number of decoder layers stacked.

D It increases the number of attention heads, allowing the model to focus on more representation subspaces.

💡 Hint

Recall what 'nhead' controls in multi-head attention.

'nhead' specifies how many parallel attention heads the model uses. More heads allow the model to attend to different parts of the input simultaneously.

🔧 Debug

advanced

Identifying error in Transformer decoder usage

What error will this code raise when running the Transformer decoder layer?

PyTorch

import torch
import torch.nn as nn

decoder_layer = nn.TransformerDecoderLayer(d_model=256, nhead=4)
memory = torch.rand(15, 16, 256)
tgt = torch.rand(15, 16, 128)  # Incorrect d_model size
output = decoder_layer(tgt, memory)

A No error, code runs successfully

B SyntaxError because of missing parentheses

C RuntimeError due to mismatched feature dimensions between tgt and memory

D TypeError because tgt is not a tensor

💡 Hint

Check the last dimension sizes of tgt and memory tensors.

The decoder layer expects tgt and memory to have the same feature dimension (d_model). Here, tgt has 128 features but memory has 256, causing a RuntimeError.

❓ Metrics

expert

Choosing the best metric for Transformer decoder language generation

Which metric is most appropriate to evaluate the quality of text generated by a Transformer decoder in a language translation task?

A BLEU score measuring n-gram overlap with reference translations

B Mean Squared Error between generated and reference token embeddings

C Accuracy of predicting the next token in the sequence

D F1 score computed on binary classification labels

💡 Hint

Think about how translation quality is commonly measured.

BLEU score compares the generated text to reference translations by measuring n-gram overlaps, which is standard for translation tasks.

Practice

1. What is the main role of the Transformer decoder in a sequence-to-sequence model?

easy

A. To reduce the dimensionality of input data

B. To encode input sequences into fixed-size vectors

C. To perform image classification tasks

D. To generate output tokens step-by-step using previous outputs and encoder info

Solution

1. Step 1: Understand the Transformer decoder purpose

   The Transformer decoder generates output sequences by attending to previous outputs and encoder outputs.

2. Step 2: Compare options with decoder role

   Only To generate output tokens step-by-step using previous outputs and encoder info describes generating output tokens step-by-step using previous outputs and encoder info, which matches the decoder's role.

3. Final Answer:

   To generate output tokens step-by-step using previous outputs and encoder info -> Option D

4. Quick Check:

   Decoder generates outputs stepwise = D [OK]

Hint: Decoder creates outputs using past outputs and encoder data [OK]

Common Mistakes:

• Confusing encoder and decoder roles
• Thinking decoder encodes inputs
• Assuming decoder reduces data size

2. Which of the following is the correct way to create a Transformer decoder layer in PyTorch?

easy

A. torch.nn.TransformerDecoderLayer(d_model=512, nhead=8)

B. torch.nn.TransformerEncoderLayer(d_model=512, nhead=8)

C. torch.nn.TransformerDecoder(d_model=512, nhead=8)

D. torch.nn.TransformerLayer(d_model=512, nhead=8)

Solution

1. Step 1: Identify the correct PyTorch class for a decoder layer

   PyTorch provides torch.nn.TransformerDecoderLayer to create a single decoder layer.

2. Step 2: Check options for correct class and parameters

   torch.nn.TransformerDecoderLayer(d_model=512, nhead=8) uses torch.nn.TransformerDecoderLayer with correct parameters. Others are either encoder layers or invalid classes.

3. Final Answer:

   torch.nn.TransformerDecoderLayer(d_model=512, nhead=8) -> Option A

4. Quick Check:

   Decoder layer class is TransformerDecoderLayer = A [OK]

Hint: Decoder layer class ends with 'DecoderLayer' in PyTorch [OK]

Common Mistakes:

• Using EncoderLayer instead of DecoderLayer
• Trying to instantiate TransformerDecoder directly
• Using non-existent class names

3. Given the code below, what is the shape of the output tensor?

import torch
from torch.nn import TransformerDecoder, TransformerDecoderLayer

layer = TransformerDecoderLayer(d_model=256, nhead=8)
decoder = TransformerDecoder(layer, num_layers=3)

memory = torch.rand(10, 32, 256)  # (source_seq_len, batch_size, d_model)
tgt = torch.rand(20, 32, 256)     # (target_seq_len, batch_size, d_model)

output = decoder(tgt, memory)

medium

A. (32, 20, 256)

B. (10, 32, 256)

C. (20, 32, 256)

D. (20, 256, 32)

Solution

1. Step 1: Understand input and output shapes for TransformerDecoder

   The decoder input shape is (target_seq_len, batch_size, d_model). The output shape matches the target input shape.

2. Step 2: Check given shapes and output

   Target input 'tgt' has shape (20, 32, 256), so output will have the same shape.

3. Final Answer:

   (20, 32, 256) -> Option C

4. Quick Check:

   Output shape matches target input shape = C [OK]

Hint: Output shape matches target input shape in decoder [OK]

Common Mistakes:

• Confusing source and target sequence lengths
• Mixing batch size and feature dimensions
• Assuming output shape changes after decoding

4. What is wrong with this PyTorch Transformer decoder usage?

import torch
from torch.nn import TransformerDecoder, TransformerDecoderLayer

layer = TransformerDecoderLayer(d_model=128, nhead=4)
decoder = TransformerDecoder(layer, num_layers=2)

memory = torch.rand(15, 16, 128)
tgt = torch.rand(15, 16, 64)  # Incorrect feature size

output = decoder(tgt, memory)

medium

A. The memory tensor has wrong batch size

B. The target tensor 'tgt' has wrong feature dimension

C. The number of decoder layers is too low

D. The nhead parameter should be equal to batch size

Solution

1. Step 1: Check feature dimensions of inputs

   The decoder expects target and memory tensors to have the same feature dimension (d_model=128).

2. Step 2: Identify mismatch in target tensor

   Target tensor 'tgt' has feature size 64, which does not match d_model=128, causing an error.

3. Final Answer:

   The target tensor 'tgt' has wrong feature dimension -> Option B

4. Quick Check:

   Feature size mismatch in target tensor = A [OK]

Hint: Check if target and memory feature sizes match d_model [OK]

Common Mistakes:

• Ignoring feature dimension mismatch
• Assuming batch size must match nhead
• Thinking number of layers causes this error

5. You want to build a Transformer decoder that generates text step-by-step using previous outputs and encoder memory. Which of the following is the best approach to implement the decoder's forward pass in PyTorch?

hard

A. Use a TransformerDecoder with masked target input to prevent attending to future tokens

B. Use a TransformerDecoder without masking so it sees all target tokens

C. Use only a TransformerEncoder for generation

D. Use a TransformerDecoder but feed the entire output sequence at once without masking

Solution

1. Step 1: Understand the need for masking in decoding

   During generation, the decoder must not look at future tokens to avoid cheating, so masking is required.

2. Step 2: Identify correct usage in PyTorch

   Using TransformerDecoder with a mask on target input prevents attending to future tokens, enabling step-by-step generation.

3. Final Answer:

   Use a TransformerDecoder with masked target input to prevent attending to future tokens -> Option A

4. Quick Check:

   Mask future tokens in decoder input = B [OK]

Hint: Mask future tokens in decoder input for stepwise generation [OK]

Common Mistakes:

• Not using masks and allowing future token attention
• Using encoder only for generation
• Feeding full output sequence without masking