__init__ for layers in PyTorch - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to define the constructor of a PyTorch layer class.

PyTorch

class MyLayer(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear([1], 10)

Drag options to blanks, or click blank then click option'

ANone

B10

C0

D5

Attempts:

3 left

💡 Hint

Common Mistakes

Using None or 0 as input size causes errors.

Confusing input and output sizes.

The input size to nn.Linear must be specified. Here, 5 is the input feature size.

2fill in blank

medium

Complete the code to initialize a convolutional layer with 3 input channels and 16 output channels.

PyTorch

class ConvLayer(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv = nn.Conv2d([1], 16, kernel_size=3)

Drag options to blanks, or click blank then click option'

A16

B32

C3

D1

Attempts:

3 left

💡 Hint

Common Mistakes

Using output channels as input channels.

Using kernel size as input channels.

The first argument to nn.Conv2d is the number of input channels, here 3 for RGB images.

3fill in blank

hard

Fix the error in the layer initialization by filling the blank correctly.

PyTorch

class CustomLayer(nn.Module):
    def __init__(self, input_dim, output_dim):
        super().__init__()
        self.fc = nn.Linear(input_dim, [1])

Drag options to blanks, or click blank then click option'

Aoutput_dim

Binput_dim

Cinput_dim * 2

DNone

Attempts:

3 left

💡 Hint

Common Mistakes

Using input_dim for both input and output sizes.

Using None as output size.

The second argument to nn.Linear is the output dimension, which should be output_dim, not input_dim.

4fill in blank

hard

Fill both blanks to initialize a dropout layer with 0.5 dropout rate and a ReLU activation.

PyTorch

class DropoutReluLayer(nn.Module):
    def __init__(self):
        super().__init__()
        self.dropout = nn.Dropout([1])
        self.activation = nn.[2]()

Drag options to blanks, or click blank then click option'

A0.5

B0.3

CReLU

DSigmoid

Attempts:

3 left

💡 Hint

Common Mistakes

Using wrong dropout rate like 0.3.

Using Sigmoid instead of ReLU.

Dropout rate is 0.5 and activation is ReLU as requested.

5fill in blank

hard

Fill all three blanks to define a layer with batch normalization, dropout, and linear transformation.

PyTorch

class ComplexLayer(nn.Module):
    def __init__(self, input_size, output_size):
        super().__init__()
        self.batchnorm = nn.BatchNorm1d([1])
        self.dropout = nn.Dropout([2])
        self.linear = nn.Linear([3], output_size)

Drag options to blanks, or click blank then click option'

Ainput_size

B0.2

Doutput_size

Attempts:

3 left

💡 Hint

Common Mistakes

Using output_size for batchnorm or linear input.

Using dropout rate greater than 1.

BatchNorm1d uses input_size, dropout rate is 0.2, and linear layer input is input_size.

Practice

1. What is the main purpose of the __init__ method in a PyTorch layer class?

easy

A. To save the model to disk

B. To set up the parts of the layer like linear or dropout layers

C. To train the model on data

D. To run the forward pass of the model

Solution

1. Step 1: Understand the role of __init__ in PyTorch

   The __init__ method is used to prepare the layer by defining its parts, such as linear or dropout layers.

2. Step 2: Differentiate from other methods

   The forward pass runs the data through the layer, training updates weights, and saving stores the model. These are not done in __init__.

3. Final Answer:

   To set up the parts of the layer like linear or dropout layers -> Option B

4. Quick Check:

   __init__ prepares layer parts = D [OK]

Hint: Remember: __init__ builds the layer parts first [OK]

Common Mistakes:

• Confusing __init__ with the forward method
• Thinking __init__ trains the model
• Assuming __init__ saves the model

2. Which of the following is the correct way to start the __init__ method in a PyTorch layer class?

easy

A. def __init__(self): super()

B. def __init__(): super().__init__()

C. def __init__(self): super().__init__()

D. def __init__(self): init.super()

Solution

1. Step 1: Check the method signature

   The __init__ method must have self as the first parameter.

2. Step 2: Call the parent class initializer correctly

   Use super().__init__() to properly initialize the parent class.

3. Final Answer:

   def __init__(self): super().__init__() -> Option C

4. Quick Check:

   Correct __init__ start = B [OK]

Hint: Always include self and call super().__init__() first [OK]

Common Mistakes:

• Omitting self parameter
• Calling super() without __init__()
• Wrong method signature without self

3. What will be the output shape of the linear layer defined below inside __init__ when input tensor has shape (10, 5)?

class MyLayer(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = torch.nn.Linear(5, 3)

    def forward(self, x):
        return self.linear(x)

layer = MyLayer()
input_tensor = torch.randn(10, 5)
output = layer(input_tensor)
print(output.shape)

medium

A. torch.Size([10, 3])

B. torch.Size([5, 3])

C. torch.Size([10, 5])

D. torch.Size([3, 10])

Solution

1. Step 1: Understand the linear layer dimensions

   The linear layer is defined as Linear(5, 3), so it expects input features of size 5 and outputs size 3.

2. Step 2: Match input tensor shape and output shape

   The input tensor has shape (10, 5), meaning 10 samples with 5 features each. The output will have shape (10, 3) after the linear layer.

3. Final Answer:

   torch.Size([10, 3]) -> Option A

4. Quick Check:

   Input features 5 to output 3, batch 10 = torch.Size([10, 3]) [OK]

Hint: Output shape = (batch size, output features) [OK]

Common Mistakes:

• Confusing input and output feature sizes
• Mixing batch size with feature dimension
• Ignoring the linear layer's output size

4. Identify the error in the following __init__ method of a PyTorch layer:

class MyLayer(torch.nn.Module):
    def __init__(self):
        self.linear = torch.nn.Linear(4, 2)
        super().__init__()

medium

A. Missing return statement in __init__

B. self.linear should be defined outside __init__

C. No error, code is correct

D. super().__init__() must be called before defining layers

Solution

1. Step 1: Check order of super().__init__() call

   In PyTorch, super().__init__() must be called first inside __init__ to properly initialize the parent class.

2. Step 2: Identify the error in the code

   The code calls super().__init__() after defining self.linear, which is incorrect and can cause errors.

3. Final Answer:

   super().__init__() must be called before defining layers -> Option D

4. Quick Check:

   Call super().__init__() first in __init__ = C [OK]

Hint: Always call super().__init__() before defining layers [OK]

Common Mistakes:

• Defining layers before calling super().__init__()
• Thinking __init__ needs a return statement
• Defining layers outside __init__

5. You want to create a custom PyTorch layer that has two linear layers and a dropout layer. Which __init__ method correctly sets up these parts?

hard

A. def __init__(self):
super().__init__()
self.linear1 = torch.nn.Linear(10, 20)
self.linear2 = torch.nn.Linear(20, 5)
self.dropout = torch.nn.Dropout(0.3)

B. def __init__(self):
self.linear1 = torch.nn.Linear(10, 20)
self.linear2 = torch.nn.Linear(20, 5)
self.dropout = torch.nn.Dropout(0.3)
super().__init__()

C. def __init__(self):
super().__init__()
self.linear1 = torch.nn.Linear(20, 10)
self.linear2 = torch.nn.Linear(5, 20)
self.dropout = torch.nn.Dropout(0.3)

D. def __init__(self):
super().__init__()
self.linear1 = torch.nn.Linear(10, 20)
self.linear2 = torch.nn.Linear(5, 20)
self.dropout = torch.nn.Dropout(0.3)

Solution

1. Step 1: Check the order of super().__init__() call

   It must be called first to initialize the parent class properly.

2. Step 2: Verify layer dimensions and definitions

   Linear layers should connect properly: first from 10 to 20, second from 20 to 5. Dropout is correctly defined with 0.3 probability.

3. Step 3: Identify the correct option

   def __init__(self): super().__init__() self.linear1 = torch.nn.Linear(10, 20) self.linear2 = torch.nn.Linear(20, 5) self.dropout = torch.nn.Dropout(0.3) matches correct order, layer sizes, and dropout setup. Codes that call super().__init__() last or have mismatched dimensions like Linear(20,10) and Linear(5,20) are incorrect.

4. Final Answer:

   def __init__(self): super().__init__() self.linear1 = torch.nn.Linear(10, 20) self.linear2 = torch.nn.Linear(20, 5) self.dropout = torch.nn.Dropout(0.3) -> Option A

5. Quick Check:

   Correct order and layer sizes = A [OK]

Hint: Call super first, then define layers with matching sizes [OK]

Common Mistakes:

• Calling super().__init__() last
• Using wrong input/output sizes for linear layers
• Misplacing dropout layer definition