Linear (fully connected) layers in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output shape of a linear layer

Given the following PyTorch code, what is the shape of the output tensor after passing input through the linear layer?

PyTorch

import torch
import torch.nn as nn

input_tensor = torch.randn(10, 5)  # batch size 10, input features 5
linear_layer = nn.Linear(5, 3)  # input features 5, output features 3
output = linear_layer(input_tensor)
print(output.shape)

A torch.Size([5, 3])

B torch.Size([10, 3])

C torch.Size([3, 10])

D torch.Size([10, 5])

💡 Hint

Remember the linear layer transforms the last dimension from input features to output features, keeping batch size the same.

The input tensor has shape (10, 5). The linear layer maps 5 input features to 3 output features. The batch size (10) remains unchanged, so output shape is (10, 3).

❓ Model Choice

intermediate

Choosing the correct linear layer for input-output sizes

You have input data with 20 features and want to output predictions with 4 values per example. Which PyTorch linear layer correctly matches this?

A nn.Linear(20, 4)

B nn.Linear(4, 20)

C nn.Linear(20, 20)

D nn.Linear(4, 4)

💡 Hint

The first argument is input features, the second is output features.

The linear layer should take 20 input features and produce 4 output features, so nn.Linear(20, 4) is correct.

❓ Hyperparameter

advanced

Effect of changing output features in a linear layer

If you increase the output features of a linear layer from 10 to 50, which of the following is true?

A The batch size of input must be reduced.

B The number of learnable parameters decreases.

C The number of learnable parameters increases.

D The number of learnable parameters stays the same.

💡 Hint

Learnable parameters depend on input and output features.

The number of parameters in a linear layer is input_features * output_features + output_features (bias). Increasing output features increases parameters.

🔧 Debug

advanced

Identifying error in linear layer input size mismatch

What error will this code raise when running the forward pass?

PyTorch

import torch
import torch.nn as nn

linear = nn.Linear(8, 4)
input_tensor = torch.randn(5, 10)
output = linear(input_tensor)

A RuntimeError: size mismatch for input features

B TypeError: input must be a tensor

C ValueError: output size must be positive

D No error, runs successfully

💡 Hint

Check if input tensor's last dimension matches linear layer's input features.

The linear layer expects input features of size 8, but input tensor has size 10 in last dimension, causing a RuntimeError.

🧠 Conceptual

expert

Why use bias in linear layers?

Which statement best explains the role of the bias term in a linear (fully connected) layer?

A Bias is used to increase the batch size during training.

B Bias reduces the number of parameters to prevent overfitting.

C Bias normalizes the input features before multiplication.

D Bias allows the model to shift the output independently of the input, enabling better fitting of data.

💡 Hint

Think about what happens if bias is zero.

Bias adds a constant term to the output, allowing the model to fit data that does not pass through the origin.