Sequence classification in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of a simple sequence classification model

What is the output shape of the logits tensor after running this PyTorch sequence classification model on a batch of 8 sequences, each of length 10, with an embedding size of 16 and 3 output classes?

PyTorch

import torch
import torch.nn as nn

class SimpleSeqClassifier(nn.Module):
    def __init__(self):
        super().__init__()
        self.embedding = nn.Embedding(20, 16)
        self.rnn = nn.GRU(16, 32, batch_first=True)
        self.fc = nn.Linear(32, 3)

    def forward(self, x):
        x = self.embedding(x)
        _, h_n = self.rnn(x)
        logits = self.fc(h_n.squeeze(0))
        return logits

model = SimpleSeqClassifier()
inputs = torch.randint(0, 20, (8, 10))
logits = model(inputs)
output_shape = logits.shape

A (32, 3)

B (8, 3)

C (8, 10, 3)

D (10, 3)

💡 Hint

The model outputs one prediction per sequence in the batch, not per time step.

The GRU returns the hidden state for the last time step with shape (1, batch_size, hidden_size). After squeezing the first dimension, it becomes (batch_size, hidden_size). The linear layer maps hidden_size to 3 classes, so the output shape is (batch_size, 3).

❓ Model Choice

intermediate

Choosing the right model for sequence classification

Which PyTorch model architecture is best suited for classifying sequences where the order of elements matters and the sequences can have variable lengths?

A A recurrent neural network (RNN) such as LSTM or GRU

B A convolutional neural network (CNN) with 1D convolutions over the sequence

C A feedforward neural network with fixed-size input vectors

D A simple linear regression model

💡 Hint

Think about models that can handle sequences of different lengths and remember order.

RNNs like LSTM or GRU are designed to process sequences of variable length and capture order dependencies, making them suitable for sequence classification tasks.

❓ Hyperparameter

advanced

Effect of hidden size on sequence classification model

In a GRU-based sequence classification model, increasing the hidden size from 32 to 128 will most likely have which effect?

A Decrease model capacity and reduce training time

B Have no effect on model capacity or training time

C Cause the model to ignore sequence order

D Increase model capacity and may improve accuracy but increase training time

💡 Hint

Hidden size controls how much information the model can store at each step.

A larger hidden size means the model can learn more complex patterns but requires more computation and memory, which increases training time.

❓ Metrics

advanced

Choosing the right metric for imbalanced sequence classification

For a sequence classification task with highly imbalanced classes, which metric is most appropriate to evaluate model performance?

A F1-score

B Accuracy

C Precision

D Mean Squared Error

💡 Hint

Consider a metric that balances false positives and false negatives.

F1-score balances precision and recall, making it suitable for imbalanced classification tasks where accuracy can be misleading.

🔧 Debug

expert

Debugging a sequence classification model with exploding gradients

You train a GRU-based sequence classification model, but the training loss suddenly becomes NaN after a few epochs. Which of the following is the most likely cause?

A The batch size is too small causing underfitting

B The model has too few layers causing underfitting

C The learning rate is too high causing exploding gradients

D The input sequences are too short causing overfitting

💡 Hint

NaN loss often happens when gradients become very large.

A high learning rate can cause gradients to explode, leading to NaN loss values during training.