forward method in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output shape after forward pass?

Given the PyTorch model below, what is the shape of the output tensor after running model(x) where x is a tensor of shape (10, 3, 32, 32)?

PyTorch

import torch
import torch.nn as nn

class SimpleCNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv = nn.Conv2d(3, 6, kernel_size=3, padding=1)
        self.pool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(6 * 16 * 16, 10)

    def forward(self, x):
        x = self.pool(torch.relu(self.conv(x)))
        x = x.view(-1, 6 * 16 * 16)
        x = self.fc(x)
        return x

model = SimpleCNN()
x = torch.randn(10, 3, 32, 32)
out = model(x)
output_shape = out.shape

A (10, 6, 16, 16)

B (6, 16, 16)

C (10, 10)

D (10, 3, 32, 32)

💡 Hint

Remember the batch size is the first dimension and the final fully connected layer outputs 10 features.

The input tensor has batch size 10. After convolution and pooling, the feature map size is (6, 16, 16). Flattening it gives 6*16*16 features per sample. The fully connected layer outputs 10 features per sample, so the output shape is (10, 10).

❓ Model Choice

intermediate

Which forward method correctly applies dropout during training only?

Choose the correct forward method implementation that applies dropout only during training in PyTorch.

PyTorch

import torch.nn as nn

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(20, 10)
        self.dropout = nn.Dropout(0.5)
        self.fc2 = nn.Linear(10, 1)

    def forward(self, x):
        # Options below
        pass

A

def forward(self, x):
    x = torch.relu(self.fc1(x))
    x = self.dropout.train()(x)
    x = self.fc2(x)
    return x

B

def forward(self, x):
    x = torch.relu(self.fc1(x))
    x = self.dropout(x)
    x = self.fc2(x)
    return x

C

def forward(self, x):
    x = torch.relu(self.fc1(x))
    x = self.dropout(x) if self.training else x
    x = self.fc2(x)
    return x

D

def forward(self, x):
    x = torch.relu(self.fc1(x))
    x = self.dropout.eval()(x)
    x = self.fc2(x)
    return x

💡 Hint

PyTorch dropout layers automatically apply dropout only during training mode.

In PyTorch, dropout layers apply dropout only when the model is in training mode. Calling dropout directly applies it correctly. Option B uses dropout normally. Option B is redundant but works similarly. Options C and A misuse eval() and train() methods on dropout, which are incorrect.

❓ Hyperparameter

advanced

Which learning rate will likely cause unstable training in this forward method?

Consider a neural network with the following forward method. Which learning rate is most likely to cause unstable training (loss exploding)?

PyTorch

import torch
import torch.nn as nn

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc = nn.Linear(100, 10)

    def forward(self, x):
        return torch.relu(self.fc(x))

model = Net()
optimizer = torch.optim.SGD(model.parameters(), lr=LR)

A 0.001

B 0.01

C 0.1

D 1.0

💡 Hint

Higher learning rates can cause the model weights to update too aggressively.

A learning rate of 1.0 is very high and will likely cause the model weights to change too much each step, causing unstable training and loss to explode. Lower rates like 0.001 or 0.01 are safer. 0.1 might be okay but risky.

🔧 Debug

advanced

Why does this forward method raise a runtime error?

Examine the forward method below. Why does it raise a runtime error when called?

PyTorch

import torch
import torch.nn as nn

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(10, 5)

    def forward(self, x):
        x = self.fc1(x)
        x = x.view(-1, 10)
        return x

model = Net()
x = torch.randn(3, 10)
out = model(x)

A The view operation tries to reshape tensor to incompatible size causing runtime error.

B The input tensor has wrong shape for the linear layer causing error.

C The forward method is missing a return statement causing None output.

D The model is missing a call to super().__init__() causing initialization error.

💡 Hint

Check the shape after the linear layer and what the view tries to reshape it into.

The linear layer outputs shape (3, 5). The view tries to reshape it to (-1, 10), which is incompatible because total elements 15 cannot be reshaped to something with 10 columns. This causes a runtime error.

🧠 Conceptual

expert

What is the role of the forward method in PyTorch's nn.Module?

Choose the best description of the role of the forward method in a PyTorch nn.Module subclass.

A It defines how input data flows through the layers to produce output during the model's forward pass.

B It initializes the model's parameters and layers before training starts.

C It updates the model's weights during backpropagation after loss calculation.

D It saves the model's state dictionary to disk for later use.

💡 Hint

Think about what happens when you call model(input).

The forward method defines the computation performed at every call of the model. It specifies how input tensors are transformed through layers to produce output. Initialization happens in __init__. Weight updates happen in the optimizer step. Saving is done by separate functions.

Practice

1. What is the main purpose of the forward method in a PyTorch model?

easy

A. To define how input data passes through the model layers

B. To initialize model weights

C. To save the model to disk

D. To load data from a dataset

Solution

1. Step 1: Understand the role of forward in PyTorch

   The forward method specifies the computation performed on input data as it passes through the model.

2. Step 2: Differentiate from other methods

   Initialization and saving/loading are handled by other methods, not forward.

3. Final Answer:

   To define how input data passes through the model layers -> Option A

4. Quick Check:

   forward defines data flow [OK]

Hint: Remember: forward = data flow through model [OK]

Common Mistakes:

• Confusing forward with initialization
• Thinking forward saves or loads data
• Assuming forward is called manually

2. Which of the following is the correct way to define the forward method inside a PyTorch nn.Module subclass?

easy

A. def Forward(self, x): return x * 2

B. def forward(self, x): return x * 2

C. def forward(x): return x * 2

D. def forward(self): return x * 2

Solution

1. Step 1: Check method signature requirements

   The forward method must have self as the first argument and input data as the second.

2. Step 2: Validate method name and parameters

   def forward(self, x): return x * 2 correctly uses lowercase forward and includes self and x. Others have wrong case, missing self, or missing input.

3. Final Answer:

   def forward(self, x): return x * 2 -> Option B

4. Quick Check:

   Correct method name and parameters = def forward(self, x): return x * 2 [OK]

Hint: Always use lowercase 'forward' with self and input args [OK]

Common Mistakes:

• Capitalizing 'Forward' instead of 'forward'
• Omitting 'self' parameter
• Missing input argument in method

3. Given the following PyTorch model code, what will be the output when calling model(torch.tensor([2.0]))?

import torch
import torch.nn as nn

class SimpleModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(1, 1)
        self.linear.weight.data.fill_(2.0)
        self.linear.bias.data.fill_(1.0)
    def forward(self, x):
        return self.linear(x)

model = SimpleModel()

medium

A. tensor([5.0])

B. tensor([4.0])

C. tensor([2.0])

D. tensor([1.0])

Solution

1. Step 1: Calculate linear layer output

   The linear layer output is weight * input + bias = 2.0 * 2.0 + 1.0 = 5.0.

2. Step 2: Confirm output tensor

   The model returns tensor([5.0]) as output.

3. Final Answer:

   tensor([5.0]) -> Option A

4. Quick Check:

   Output = 2*2 + 1 = 5 [OK]

Hint: Output = weight * input + bias in linear layer [OK]

Common Mistakes:

• Ignoring bias term
• Using wrong weight value
• Confusing input tensor shape

4. Identify the error in this PyTorch model's forward method:

class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc = nn.Linear(3, 1)
    def forward(self, x):
        x = self.fc
        return x

medium

A. Input tensor shape is incorrect

B. Missing call to super().__init__() in __init__

C. The forward method should not have a return statement

D. The forward method returns the layer object, not the output

Solution

1. Step 1: Analyze the forward method code

   The line x = self.fc assigns the layer object to x instead of calling it.

2. Step 2: Identify correct usage

   To get output, the layer must be called with input: x = self.fc(x).

3. Final Answer:

   The forward method returns the layer object, not the output -> Option D

4. Quick Check:

   Layer must be called with input, not assigned [OK]

Hint: Call layers with input inside forward, don't assign layer itself [OK]

Common Mistakes:

• Assigning layer instead of calling it
• Forgetting to call super().__init__()
• Ignoring input tensor shape

5. You want to create a PyTorch model that applies two linear layers sequentially in the forward method, with a ReLU activation in between. Which forward method implementation is correct?

class TwoLayerModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(4, 3)
        self.fc2 = nn.Linear(3, 2)
        self.relu = nn.ReLU()

    def forward(self, x):
        ?

hard

A. x = self.fc1(x)
x = self.fc2(x)
x = self.relu(x)
return x

B. x = self.relu(self.fc1(x))
return self.fc2(x)

C. x = self.fc1(x)
x = self.relu(x)
x = self.fc2(x)
return x

D. x = self.fc1(x)
return self.fc2(self.relu(x))

Solution

1. Step 1: Understand the desired layer order

   The model should apply fc1, then ReLU, then fc2 in sequence.

2. Step 2: Check each option's order

   x = self.fc1(x) x = self.relu(x) x = self.fc2(x) return x correctly applies fc1, then ReLU, then fc2. Others change the order or skip activation between layers.

3. Final Answer:

   x = self.fc1(x) x = self.relu(x) x = self.fc2(x) return x -> Option C

4. Quick Check:

   Layer1 -> ReLU -> Layer2 sequence = x = self.fc1(x) x = self.relu(x) x = self.fc2(x) return x [OK]

Hint: Apply activation between layers, not after last layer [OK]

Common Mistakes:

• Applying activation after last layer
• Skipping activation between layers
• Incorrect order of operations