Forward pass computation in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of a simple forward pass in PyTorch

What is the output tensor after running this forward pass code?

PyTorch

import torch
import torch.nn as nn

class SimpleModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(2, 1, bias=False)

    def forward(self, x):
        return self.linear(x)

model = SimpleModel()
with torch.no_grad():
    model.linear.weight.copy_(torch.tensor([[2.0, 3.0]]))
    input_tensor = torch.tensor([[1.0, 2.0]])
    output = model(input_tensor)
    print(output)

A tensor([[8.]])

B tensor([[7.]])

C tensor([[5.]])

D tensor([[10.]])

💡 Hint

Remember the forward pass multiplies input by weights and sums them.

The linear layer multiplies input [1, 2] by weights [2, 3] without bias: 1*2 + 2*3 = 8.

❓ Model Choice

intermediate

Choosing the correct model output shape

Given a PyTorch model with nn.Linear(4, 3) and input tensor shape (5, 4), what is the output shape after the forward pass?

A (5, 4)

B (4, 3)

C (3, 5)

D (5, 3)

💡 Hint

The linear layer changes the last dimension from input features to output features.

The input batch size is 5, each with 4 features. The linear layer outputs 3 features per input, so output shape is (5, 3).

❓ Hyperparameter

advanced

Effect of activation function in forward pass

Which activation function will output only values between 0 and 1 during the forward pass?

A ReLU

B Sigmoid

C Tanh

D LeakyReLU

💡 Hint

Think about which function squashes outputs into a probability-like range.

Sigmoid outputs values strictly between 0 and 1, suitable for probabilities.

❓ Metrics

advanced

Calculating accuracy from model predictions

Given model predictions tensor([0, 1, 1, 0]) and true labels tensor([0, 0, 1, 0]), what is the accuracy?

A 0.75

B 0.5

C 0.25

D 1.0

💡 Hint

Accuracy is number of correct predictions divided by total predictions.

Correct predictions are at indices 0, 2, and 3 (3 out of 4), so accuracy is 0.75.

🔧 Debug

expert

Identifying error in forward pass code

What error will this PyTorch forward pass code raise? import torch import torch.nn as nn class Model(nn.Module): def __init__(self): super().__init__() self.linear = nn.Linear(3, 2) def forward(self, x): return self.linear(x) model = Model() input_tensor = torch.randn(4, 4) output = model(input_tensor)

PyTorch

import torch
import torch.nn as nn

class Model(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(3, 2)

    def forward(self, x):
        return self.linear(x)

model = Model()
input_tensor = torch.randn(4, 4)
output = model(input_tensor)

A No error, outputs tensor of shape (4, 2)

B TypeError: forward() missing 1 required positional argument

C RuntimeError: size mismatch

D ValueError: input tensor must be 1D

💡 Hint

Check if input tensor shape matches the expected input features of the linear layer.

The linear layer expects input with last dimension 3, but input tensor has last dimension 4, causing size mismatch error.

Practice

1. What does the forward pass in a PyTorch model do?

easy

A. It moves input data through the model to get predictions.

B. It updates the model weights during training.

C. It saves the model to disk.

D. It initializes the model parameters.

Solution

1. Step 1: Understand the role of forward pass

   The forward pass takes input data and processes it through the model layers to produce output predictions.

2. Step 2: Differentiate from other processes

   Updating weights happens in backward pass, saving is separate, and initialization is before training.

3. Final Answer:

   It moves input data through the model to get predictions. -> Option A

4. Quick Check:

   Forward pass = data to predictions [OK]

Hint: Forward pass = input to output predictions [OK]

Common Mistakes:

• Confusing forward pass with weight update
• Thinking forward pass saves the model
• Mixing initialization with forward pass

2. Which of the following is the correct way to run a forward pass on a PyTorch model named model with input tensor x?

easy

A. output = model.forward(x)

B. output = model.predict(x)

C. output = model.run(x)

D. output = model(x)

Solution

1. Step 1: Recall PyTorch model call syntax

   In PyTorch, calling model(x) runs the forward pass internally.

2. Step 2: Identify incorrect options

   Directly calling model.forward(x) is discouraged; model.run and model.predict are not standard methods.

3. Final Answer:

   output = model(x) -> Option D

4. Quick Check:

   Use model(x) to run forward pass [OK]

Hint: Use model(input) to run forward pass [OK]

Common Mistakes:

• Calling model.forward(x) directly
• Using non-existent methods like model.run or model.predict
• Confusing syntax with other frameworks

3. Given the following PyTorch code, what is the output tensor shape?

import torch
import torch.nn as nn

class SimpleNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(4, 2)
    def forward(self, x):
        return self.linear(x)

model = SimpleNet()
x = torch.randn(3, 4)
output = model(x)
print(output.shape)

medium

A. torch.Size([3, 2])

B. torch.Size([2, 3])

C. torch.Size([4, 2])

D. torch.Size([3, 4])

Solution

1. Step 1: Analyze input and layer dimensions

   The input tensor has shape (3, 4), meaning batch size 3 and feature size 4. The linear layer maps 4 features to 2 outputs.

2. Step 2: Compute output shape

   Applying linear layer to each batch item changes feature size from 4 to 2, so output shape is (3, 2).

3. Final Answer:

   torch.Size([3, 2]) -> Option A

4. Quick Check:

   Input batch 3, features 4 -> output features 2 [OK]

Hint: Output shape = (batch, output features) [OK]

Common Mistakes:

• Mixing batch and feature dimensions
• Assuming output shape equals input shape
• Confusing input and output feature sizes

4. Identify the error in this PyTorch forward pass code snippet:

import torch
import torch.nn as nn

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc = nn.Linear(5, 3)
    def forward(self, x):
        return self.fc(x)

model = Net()
x = torch.randn(4, 4)
output = model(x)
print(output)

medium

A. Missing activation function after Linear layer.

B. Model is not moved to GPU before forward pass.

C. Input tensor shape does not match Linear layer input size.

D. Forward method is missing return statement.

Solution

1. Step 1: Check input tensor shape vs Linear layer input

   The Linear layer expects input features of size 5, but input tensor has shape (4, 4), so feature size is 4, which mismatches.

2. Step 2: Confirm other options

   GPU usage is optional, activation functions are not mandatory, and forward method correctly returns output.

3. Final Answer:

   Input tensor shape does not match Linear layer input size. -> Option C

4. Quick Check:

   Input features must match Linear input size [OK]

Hint: Check input feature size matches Linear layer input [OK]

Common Mistakes:

• Ignoring input size mismatch errors
• Assuming GPU is required for forward pass
• Thinking activation is always needed

5. You have a PyTorch model with two linear layers chained in the forward pass:

class ChainNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(3, 4)
        self.fc2 = nn.Linear(4, 2)
    def forward(self, x):
        x = self.fc1(x)
        x = torch.relu(x)
        x = self.fc2(x)
        return x

model = ChainNet()
x = torch.tensor([[1.0, 2.0, 3.0]])
output = model(x)
print(output)

What is the shape of output and why?

hard

A. torch.Size([1, 4]) because the first layer outputs 4 features.

B. torch.Size([1, 2]) because input batch size is 1 and final layer outputs 2 features.

C. torch.Size([3, 2]) because input has 3 features and output has 2 features.

D. torch.Size([4, 2]) because the two layers output sizes multiply.

Solution

1. Step 1: Analyze input and layer sizes

   Input tensor shape is (1, 3) -- batch size 1, feature size 3. First layer maps 3 to 4 features, second layer maps 4 to 2 features.

2. Step 2: Determine output shape after forward pass

   After passing through both layers, output shape is (1, 2) because batch size stays the same and final layer outputs 2 features.

3. Final Answer:

   torch.Size([1, 2]) because input batch size is 1 and final layer outputs 2 features. -> Option B

4. Quick Check:

   Batch size unchanged, output features = final layer output [OK]

Hint: Output shape = (batch size, last layer output features) [OK]

Common Mistakes:

• Multiplying output sizes instead of chaining
• Confusing batch size with feature size
• Ignoring activation does not change shape