Sequence classification in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of a simple sequence classification model

What is the output shape of the logits tensor after running this PyTorch sequence classification model on a batch of 8 sequences, each of length 10, with an embedding size of 16 and 3 output classes?

PyTorch

import torch
import torch.nn as nn

class SimpleSeqClassifier(nn.Module):
    def __init__(self):
        super().__init__()
        self.embedding = nn.Embedding(20, 16)
        self.rnn = nn.GRU(16, 32, batch_first=True)
        self.fc = nn.Linear(32, 3)

    def forward(self, x):
        x = self.embedding(x)
        _, h_n = self.rnn(x)
        logits = self.fc(h_n.squeeze(0))
        return logits

model = SimpleSeqClassifier()
inputs = torch.randint(0, 20, (8, 10))
logits = model(inputs)
output_shape = logits.shape

A (32, 3)

B (8, 3)

C (8, 10, 3)

D (10, 3)

💡 Hint

The model outputs one prediction per sequence in the batch, not per time step.

The GRU returns the hidden state for the last time step with shape (1, batch_size, hidden_size). After squeezing the first dimension, it becomes (batch_size, hidden_size). The linear layer maps hidden_size to 3 classes, so the output shape is (batch_size, 3).

❓ Model Choice

intermediate

Choosing the right model for sequence classification

Which PyTorch model architecture is best suited for classifying sequences where the order of elements matters and the sequences can have variable lengths?

A A recurrent neural network (RNN) such as LSTM or GRU

B A convolutional neural network (CNN) with 1D convolutions over the sequence

C A feedforward neural network with fixed-size input vectors

D A simple linear regression model

💡 Hint

Think about models that can handle sequences of different lengths and remember order.

RNNs like LSTM or GRU are designed to process sequences of variable length and capture order dependencies, making them suitable for sequence classification tasks.

❓ Hyperparameter

advanced

Effect of hidden size on sequence classification model

In a GRU-based sequence classification model, increasing the hidden size from 32 to 128 will most likely have which effect?

A Decrease model capacity and reduce training time

B Have no effect on model capacity or training time

C Cause the model to ignore sequence order

D Increase model capacity and may improve accuracy but increase training time

💡 Hint

Hidden size controls how much information the model can store at each step.

A larger hidden size means the model can learn more complex patterns but requires more computation and memory, which increases training time.

❓ Metrics

advanced

Choosing the right metric for imbalanced sequence classification

For a sequence classification task with highly imbalanced classes, which metric is most appropriate to evaluate model performance?

A F1-score

B Accuracy

C Precision

D Mean Squared Error

💡 Hint

Consider a metric that balances false positives and false negatives.

F1-score balances precision and recall, making it suitable for imbalanced classification tasks where accuracy can be misleading.

🔧 Debug

expert

Debugging a sequence classification model with exploding gradients

You train a GRU-based sequence classification model, but the training loss suddenly becomes NaN after a few epochs. Which of the following is the most likely cause?

A The batch size is too small causing underfitting

B The model has too few layers causing underfitting

C The learning rate is too high causing exploding gradients

D The input sequences are too short causing overfitting

💡 Hint

NaN loss often happens when gradients become very large.

A high learning rate can cause gradients to explode, leading to NaN loss values during training.

Practice

1. What is the main goal of sequence classification in PyTorch?

easy

A. To assign a label to the entire input sequence

B. To predict the next item in the sequence

C. To label each item in the sequence separately

D. To generate a new sequence from the input

Solution

1. Step 1: Understand sequence classification

   Sequence classification means giving one label to the whole sequence, not to individual items.

2. Step 2: Compare options

   Only To assign a label to the entire input sequence describes labeling the entire sequence, which matches the goal of sequence classification.

3. Final Answer:

   To assign a label to the entire input sequence -> Option A

4. Quick Check:

   Sequence classification = label whole sequence [OK]

Hint: Sequence classification labels the whole sequence, not parts [OK]

Common Mistakes:

• Confusing sequence classification with sequence labeling
• Thinking it predicts next sequence item
• Assuming it generates new sequences

2. Which PyTorch module is commonly used to process sequences step-by-step for classification?

easy

A. torch.nn.Conv2d

B. torch.nn.Linear

C. torch.nn.RNN

D. torch.nn.BatchNorm1d

Solution

1. Step 1: Identify sequence processing modules

   RNN (Recurrent Neural Network) modules process sequences step-by-step, capturing order.

2. Step 2: Match options to sequence processing

   Only torch.nn.RNN is designed for sequential data; others serve different purposes.

3. Final Answer:

   torch.nn.RNN -> Option C

4. Quick Check:

   RNN processes sequences stepwise [OK]

Hint: RNN modules handle sequences stepwise in PyTorch [OK]

Common Mistakes:

• Choosing Linear which is for fixed-size input
• Selecting Conv2d meant for images
• Picking BatchNorm which normalizes features

3. Given this PyTorch code snippet for sequence classification, what is the shape of the output tensor?

rnn = torch.nn.RNN(input_size=10, hidden_size=20, batch_first=True)
inputs = torch.randn(5, 7, 10)  # batch=5, seq_len=7, features=10
output, hn = rnn(inputs)
final_output = hn.squeeze(0)

medium

A. [5, 20]

B. [5, 7, 20]

C. [7, 20]

D. [5, 10]

Solution

1. Step 1: Understand RNN output shapes

   Output shape is (batch, seq_len, hidden_size) = (5,7,20). hn shape is (num_layers, batch, hidden_size) = (1,5,20).

2. Step 2: Analyze final_output shape

   hn.squeeze(0) removes the first dimension (num_layers), resulting in (5,20).

3. Final Answer:

   [5, 20] -> Option A

4. Quick Check:

   hn.squeeze(0) shape = [batch, hidden_size] = [5, 20] [OK]

Hint: Squeeze removes layer dim; output shape is batch x hidden size [OK]

Common Mistakes:

• Confusing output and hn shapes
• Not squeezing the layer dimension
• Mixing sequence length with batch size

4. Identify the error in this PyTorch sequence classification model code:

class SeqClassifier(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.rnn = torch.nn.RNN(10, 20, batch_first=True)
        self.fc = torch.nn.Linear(10, 2)
    def forward(self, x):
        out, hn = self.rnn(x)
        out = self.fc(hn.squeeze(0))
        return out

medium

A. The forward method should return hn, not out

B. The RNN input size should be 2, not 10

C. The squeeze(0) should be applied to out, not hn

D. The Linear layer input size should be 20, not 10

Solution

1. Step 1: Check Linear layer input size

   The RNN hidden size is 20, so hn has shape (batch, 20). The Linear layer expects input size 10, which is incorrect.

2. Step 2: Correct Linear input size

   Linear layer input size must match hidden size 20 to process hn correctly.

3. Final Answer:

   The Linear layer input size should be 20, not 10 -> Option D

4. Quick Check:

   Linear input size = hidden size = 20 [OK]

Hint: Linear input size must match RNN hidden size [OK]

Common Mistakes:

• Mismatching Linear input size with hidden size
• Applying squeeze to wrong tensor
• Returning wrong tensor from forward

5. You want to classify sequences of varying lengths using an RNN in PyTorch. Which approach correctly handles different sequence lengths during training?

hard

A. Truncate all sequences to the shortest length without padding

B. Pad sequences to the same length and use pack_padded_sequence before RNN

C. Feed sequences directly without padding or packing

D. Use a Linear layer instead of RNN to avoid sequence length issues

Solution

1. Step 1: Understand variable-length sequence handling

   Sequences must be padded to the same length for batch processing, then packed to ignore padding during RNN.

2. Step 2: Evaluate options

   Pad sequences to the same length and use pack_padded_sequence before RNN uses padding plus pack_padded_sequence, the correct PyTorch method to handle varying lengths efficiently.

3. Final Answer:

   Pad sequences to the same length and use pack_padded_sequence before RNN -> Option B

4. Quick Check:

   Use padding + pack_padded_sequence for variable lengths [OK]

Hint: Pad then pack sequences to handle varying lengths in RNN [OK]

Common Mistakes:

• Ignoring padding and feeding raw sequences
• Truncating sequences losing data
• Replacing RNN with Linear layer incorrectly