__init__ for layers in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of a simple PyTorch layer initialization

What is the output of the following code snippet that initializes a linear layer and prints its weight shape?

PyTorch

import torch
import torch.nn as nn

layer = nn.Linear(10, 5)
print(layer.weight.shape)

A torch.Size([10, 10])

B torch.Size([10, 5])

C torch.Size([5, 10])

D torch.Size([5, 5])

💡 Hint

Remember that nn.Linear(in_features, out_features) creates weights of shape (out_features, in_features).

The weight matrix in nn.Linear has shape (out_features, in_features). Here, in_features=10 and out_features=5, so the shape is (5, 10).

❓ Model Choice

intermediate

Choosing the correct __init__ signature for a convolutional layer

Which of the following __init__ method signatures correctly initializes a 2D convolutional layer with 3 input channels, 16 output channels, and a kernel size of 3?

A

def __init__(self):
    super().__init__()
    self.conv = nn.Linear(3, 16)

B

def __init__(self):
    super().__init__()
    self.conv = nn.Conv1d(3, 16, 3)

C

def __init__(self):
    super().__init__()
    self.conv = nn.Conv2d(16, 3, 3)

D

def __init__(self):
    super().__init__()
    self.conv = nn.Conv2d(3, 16, 3)

💡 Hint

Conv2d expects (in_channels, out_channels, kernel_size).

nn.Conv2d takes input channels first, then output channels, then kernel size. Option D matches this order with correct values.

❓ Hyperparameter

advanced

Effect of bias parameter in layer initialization

In PyTorch, when initializing a linear layer with nn.Linear(10, 5, bias=False), what is the effect of setting bias=False?

A The layer will initialize bias with zeros.

B The layer will not have a bias term added to the output.

C The layer will raise an error because bias is required.

D The layer will add a bias term initialized randomly.

💡 Hint

Bias controls whether an extra constant term is added to the output.

Setting bias=False disables the bias term, so the layer output is just a weighted sum without an added constant.

🔧 Debug

advanced

Identify the error in this layer __init__ method

What error will occur when running this code snippet?

PyTorch

import torch.nn as nn

class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(10, 5)

A AttributeError because super().__init__() is called after layer initialization

B SyntaxError due to missing colon

C TypeError because nn.Linear requires extra arguments

D No error, code runs fine

💡 Hint

In PyTorch, always call super().__init__() before defining layers.

super().__init__() must be called before defining layers to properly initialize the base class. Calling it after causes attribute errors.

🧠 Conceptual

expert

Understanding parameter registration in __init__

Why is it important to define layers as attributes inside the __init__ method of a PyTorch nn.Module subclass?

A Because only attributes defined in __init__ are registered as model parameters and updated during training.

B Because layers defined outside __init__ run faster during training.

C Because layers defined in __init__ cannot be saved or loaded.

D Because layers defined in __init__ do not require calling super().__init__().

💡 Hint

Think about how PyTorch tracks parameters for optimization.

PyTorch registers parameters only if they are assigned as attributes inside __init__. This allows optimizer to find and update them.