__init__ for layers in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of a simple PyTorch layer initialization

What is the output of the following code snippet that initializes a linear layer and prints its weight shape?

PyTorch

import torch
import torch.nn as nn

layer = nn.Linear(10, 5)
print(layer.weight.shape)

A torch.Size([10, 10])

B torch.Size([10, 5])

C torch.Size([5, 10])

D torch.Size([5, 5])

💡 Hint

Remember that nn.Linear(in_features, out_features) creates weights of shape (out_features, in_features).

The weight matrix in nn.Linear has shape (out_features, in_features). Here, in_features=10 and out_features=5, so the shape is (5, 10).

❓ Model Choice

intermediate

Choosing the correct __init__ signature for a convolutional layer

Which of the following __init__ method signatures correctly initializes a 2D convolutional layer with 3 input channels, 16 output channels, and a kernel size of 3?

A

def __init__(self):
    super().__init__()
    self.conv = nn.Linear(3, 16)

B

def __init__(self):
    super().__init__()
    self.conv = nn.Conv1d(3, 16, 3)

C

def __init__(self):
    super().__init__()
    self.conv = nn.Conv2d(16, 3, 3)

D

def __init__(self):
    super().__init__()
    self.conv = nn.Conv2d(3, 16, 3)

💡 Hint

Conv2d expects (in_channels, out_channels, kernel_size).

nn.Conv2d takes input channels first, then output channels, then kernel size. Option D matches this order with correct values.

❓ Hyperparameter

advanced

Effect of bias parameter in layer initialization

In PyTorch, when initializing a linear layer with nn.Linear(10, 5, bias=False), what is the effect of setting bias=False?

A The layer will initialize bias with zeros.

B The layer will not have a bias term added to the output.

C The layer will raise an error because bias is required.

D The layer will add a bias term initialized randomly.

💡 Hint

Bias controls whether an extra constant term is added to the output.

Setting bias=False disables the bias term, so the layer output is just a weighted sum without an added constant.

🔧 Debug

advanced

Identify the error in this layer __init__ method

What error will occur when running this code snippet?

PyTorch

import torch.nn as nn

class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(10, 5)

A AttributeError because super().__init__() is called after layer initialization

B SyntaxError due to missing colon

C TypeError because nn.Linear requires extra arguments

D No error, code runs fine

💡 Hint

In PyTorch, always call super().__init__() before defining layers.

super().__init__() must be called before defining layers to properly initialize the base class. Calling it after causes attribute errors.

🧠 Conceptual

expert

Understanding parameter registration in __init__

Why is it important to define layers as attributes inside the __init__ method of a PyTorch nn.Module subclass?

A Because only attributes defined in __init__ are registered as model parameters and updated during training.

B Because layers defined outside __init__ run faster during training.

C Because layers defined in __init__ cannot be saved or loaded.

D Because layers defined in __init__ do not require calling super().__init__().

💡 Hint

Think about how PyTorch tracks parameters for optimization.

PyTorch registers parameters only if they are assigned as attributes inside __init__. This allows optimizer to find and update them.

Practice

1. What is the main purpose of the __init__ method in a PyTorch layer class?

easy

A. To save the model to disk

B. To set up the parts of the layer like linear or dropout layers

C. To train the model on data

D. To run the forward pass of the model

Solution

1. Step 1: Understand the role of __init__ in PyTorch

   The __init__ method is used to prepare the layer by defining its parts, such as linear or dropout layers.

2. Step 2: Differentiate from other methods

   The forward pass runs the data through the layer, training updates weights, and saving stores the model. These are not done in __init__.

3. Final Answer:

   To set up the parts of the layer like linear or dropout layers -> Option B

4. Quick Check:

   __init__ prepares layer parts = D [OK]

Hint: Remember: __init__ builds the layer parts first [OK]

Common Mistakes:

• Confusing __init__ with the forward method
• Thinking __init__ trains the model
• Assuming __init__ saves the model

2. Which of the following is the correct way to start the __init__ method in a PyTorch layer class?

easy

A. def __init__(self): super()

B. def __init__(): super().__init__()

C. def __init__(self): super().__init__()

D. def __init__(self): init.super()

Solution

1. Step 1: Check the method signature

   The __init__ method must have self as the first parameter.

2. Step 2: Call the parent class initializer correctly

   Use super().__init__() to properly initialize the parent class.

3. Final Answer:

   def __init__(self): super().__init__() -> Option C

4. Quick Check:

   Correct __init__ start = B [OK]

Hint: Always include self and call super().__init__() first [OK]

Common Mistakes:

• Omitting self parameter
• Calling super() without __init__()
• Wrong method signature without self

3. What will be the output shape of the linear layer defined below inside __init__ when input tensor has shape (10, 5)?

class MyLayer(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = torch.nn.Linear(5, 3)

    def forward(self, x):
        return self.linear(x)

layer = MyLayer()
input_tensor = torch.randn(10, 5)
output = layer(input_tensor)
print(output.shape)

medium

A. torch.Size([10, 3])

B. torch.Size([5, 3])

C. torch.Size([10, 5])

D. torch.Size([3, 10])

Solution

1. Step 1: Understand the linear layer dimensions

   The linear layer is defined as Linear(5, 3), so it expects input features of size 5 and outputs size 3.

2. Step 2: Match input tensor shape and output shape

   The input tensor has shape (10, 5), meaning 10 samples with 5 features each. The output will have shape (10, 3) after the linear layer.

3. Final Answer:

   torch.Size([10, 3]) -> Option A

4. Quick Check:

   Input features 5 to output 3, batch 10 = torch.Size([10, 3]) [OK]

Hint: Output shape = (batch size, output features) [OK]

Common Mistakes:

• Confusing input and output feature sizes
• Mixing batch size with feature dimension
• Ignoring the linear layer's output size

4. Identify the error in the following __init__ method of a PyTorch layer:

class MyLayer(torch.nn.Module):
    def __init__(self):
        self.linear = torch.nn.Linear(4, 2)
        super().__init__()

medium

A. Missing return statement in __init__

B. self.linear should be defined outside __init__

C. No error, code is correct

D. super().__init__() must be called before defining layers

Solution

1. Step 1: Check order of super().__init__() call

   In PyTorch, super().__init__() must be called first inside __init__ to properly initialize the parent class.

2. Step 2: Identify the error in the code

   The code calls super().__init__() after defining self.linear, which is incorrect and can cause errors.

3. Final Answer:

   super().__init__() must be called before defining layers -> Option D

4. Quick Check:

   Call super().__init__() first in __init__ = C [OK]

Hint: Always call super().__init__() before defining layers [OK]

Common Mistakes:

• Defining layers before calling super().__init__()
• Thinking __init__ needs a return statement
• Defining layers outside __init__

5. You want to create a custom PyTorch layer that has two linear layers and a dropout layer. Which __init__ method correctly sets up these parts?

hard

A. def __init__(self):
super().__init__()
self.linear1 = torch.nn.Linear(10, 20)
self.linear2 = torch.nn.Linear(20, 5)
self.dropout = torch.nn.Dropout(0.3)

B. def __init__(self):
self.linear1 = torch.nn.Linear(10, 20)
self.linear2 = torch.nn.Linear(20, 5)
self.dropout = torch.nn.Dropout(0.3)
super().__init__()

C. def __init__(self):
super().__init__()
self.linear1 = torch.nn.Linear(20, 10)
self.linear2 = torch.nn.Linear(5, 20)
self.dropout = torch.nn.Dropout(0.3)

D. def __init__(self):
super().__init__()
self.linear1 = torch.nn.Linear(10, 20)
self.linear2 = torch.nn.Linear(5, 20)
self.dropout = torch.nn.Dropout(0.3)

Solution

1. Step 1: Check the order of super().__init__() call

   It must be called first to initialize the parent class properly.

2. Step 2: Verify layer dimensions and definitions

   Linear layers should connect properly: first from 10 to 20, second from 20 to 5. Dropout is correctly defined with 0.3 probability.

3. Step 3: Identify the correct option

   def __init__(self): super().__init__() self.linear1 = torch.nn.Linear(10, 20) self.linear2 = torch.nn.Linear(20, 5) self.dropout = torch.nn.Dropout(0.3) matches correct order, layer sizes, and dropout setup. Codes that call super().__init__() last or have mismatched dimensions like Linear(20,10) and Linear(5,20) are incorrect.

4. Final Answer:

   def __init__(self): super().__init__() self.linear1 = torch.nn.Linear(10, 20) self.linear2 = torch.nn.Linear(20, 5) self.dropout = torch.nn.Dropout(0.3) -> Option A

5. Quick Check:

   Correct order and layer sizes = A [OK]

Hint: Call super first, then define layers with matching sizes [OK]

Common Mistakes:

• Calling super().__init__() last
• Using wrong input/output sizes for linear layers
• Misplacing dropout layer definition