forward method in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output shape after forward pass?

Given the PyTorch model below, what is the shape of the output tensor after running model(x) where x is a tensor of shape (10, 3, 32, 32)?

PyTorch

import torch
import torch.nn as nn

class SimpleCNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv = nn.Conv2d(3, 6, kernel_size=3, padding=1)
        self.pool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(6 * 16 * 16, 10)

    def forward(self, x):
        x = self.pool(torch.relu(self.conv(x)))
        x = x.view(-1, 6 * 16 * 16)
        x = self.fc(x)
        return x

model = SimpleCNN()
x = torch.randn(10, 3, 32, 32)
out = model(x)
output_shape = out.shape

A (10, 6, 16, 16)

B (6, 16, 16)

C (10, 10)

D (10, 3, 32, 32)

💡 Hint

Remember the batch size is the first dimension and the final fully connected layer outputs 10 features.

The input tensor has batch size 10. After convolution and pooling, the feature map size is (6, 16, 16). Flattening it gives 6*16*16 features per sample. The fully connected layer outputs 10 features per sample, so the output shape is (10, 10).

❓ Model Choice

intermediate

Which forward method correctly applies dropout during training only?

Choose the correct forward method implementation that applies dropout only during training in PyTorch.

PyTorch

import torch.nn as nn

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(20, 10)
        self.dropout = nn.Dropout(0.5)
        self.fc2 = nn.Linear(10, 1)

    def forward(self, x):
        # Options below
        pass

A

def forward(self, x):
    x = torch.relu(self.fc1(x))
    x = self.dropout.train()(x)
    x = self.fc2(x)
    return x

B

def forward(self, x):
    x = torch.relu(self.fc1(x))
    x = self.dropout(x)
    x = self.fc2(x)
    return x

C

def forward(self, x):
    x = torch.relu(self.fc1(x))
    x = self.dropout(x) if self.training else x
    x = self.fc2(x)
    return x

D

def forward(self, x):
    x = torch.relu(self.fc1(x))
    x = self.dropout.eval()(x)
    x = self.fc2(x)
    return x

💡 Hint

PyTorch dropout layers automatically apply dropout only during training mode.

In PyTorch, dropout layers apply dropout only when the model is in training mode. Calling dropout directly applies it correctly. Option B uses dropout normally. Option B is redundant but works similarly. Options C and A misuse eval() and train() methods on dropout, which are incorrect.

❓ Hyperparameter

advanced

Which learning rate will likely cause unstable training in this forward method?

Consider a neural network with the following forward method. Which learning rate is most likely to cause unstable training (loss exploding)?

PyTorch

import torch
import torch.nn as nn

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc = nn.Linear(100, 10)

    def forward(self, x):
        return torch.relu(self.fc(x))

model = Net()
optimizer = torch.optim.SGD(model.parameters(), lr=LR)

A 0.001

B 0.01

C 0.1

D 1.0

💡 Hint

Higher learning rates can cause the model weights to update too aggressively.

A learning rate of 1.0 is very high and will likely cause the model weights to change too much each step, causing unstable training and loss to explode. Lower rates like 0.001 or 0.01 are safer. 0.1 might be okay but risky.

🔧 Debug

advanced

Why does this forward method raise a runtime error?

Examine the forward method below. Why does it raise a runtime error when called?

PyTorch

import torch
import torch.nn as nn

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(10, 5)

    def forward(self, x):
        x = self.fc1(x)
        x = x.view(-1, 10)
        return x

model = Net()
x = torch.randn(3, 10)
out = model(x)

A The view operation tries to reshape tensor to incompatible size causing runtime error.

B The input tensor has wrong shape for the linear layer causing error.

C The forward method is missing a return statement causing None output.

D The model is missing a call to super().__init__() causing initialization error.

💡 Hint

Check the shape after the linear layer and what the view tries to reshape it into.

The linear layer outputs shape (3, 5). The view tries to reshape it to (-1, 10), which is incompatible because total elements 15 cannot be reshaped to something with 10 columns. This causes a runtime error.

🧠 Conceptual

expert

What is the role of the forward method in PyTorch's nn.Module?

Choose the best description of the role of the forward method in a PyTorch nn.Module subclass.

A It defines how input data flows through the layers to produce output during the model's forward pass.

B It initializes the model's parameters and layers before training starts.

C It updates the model's weights during backpropagation after loss calculation.

D It saves the model's state dictionary to disk for later use.

💡 Hint

Think about what happens when you call model(input).

The forward method defines the computation performed at every call of the model. It specifies how input tensors are transformed through layers to produce output. Initialization happens in __init__. Weight updates happen in the optimizer step. Saving is done by separate functions.