__init__ for layers in PyTorch - Model Metrics & Evaluation

Metrics & Evaluation - __init__ for layers

Which metric matters for __init__ for layers and WHY

The __init__ method in PyTorch layers sets up the parts of the model that learn, like weights and biases. While __init__ itself doesn't produce metrics, the way you define layers affects the model's ability to learn well. So, metrics like loss and accuracy during training show if your layer setup is good.

For example, if you forget to initialize a layer properly, the model might not learn, and loss won't improve. So, the key metrics to watch after defining layers are training loss and validation accuracy.

Confusion matrix or equivalent visualization

Since __init__ is about setting up layers, it doesn't directly produce predictions or confusion matrices. But once the model is trained, you can use a confusion matrix to see how well the model predicts classes.

Confusion Matrix Example:

          Predicted
          0    1
Actual 0 50   10
       1  5   35

TP = 35, FP = 10, TN = 50, FN = 5
    

This matrix helps calculate precision and recall, which tell you how well your model (built with your layers) is doing.

Precision vs Recall tradeoff with concrete examples

When your layers are set up well, your model balances precision and recall nicely. But if layers are too simple or too complex, this balance can break.

Example: Imagine a spam email detector built with layers defined in __init__. If the model has high precision but low recall, it means it catches only very obvious spam but misses many spam emails. If recall is high but precision is low, it marks many good emails as spam.

Good layer design helps the model learn features that balance precision and recall well.

What "good" vs "bad" metric values look like for this use case

Good layer initialization leads to:

• Training loss steadily decreasing
• Validation accuracy improving and stabilizing
• Balanced precision and recall (e.g., both above 0.8 in classification)

Bad layer initialization or design causes:

• Loss stuck high or not decreasing
• Validation accuracy low or fluctuating wildly
• Precision or recall very low (e.g., below 0.5), showing poor predictions

Metrics pitfalls

• Ignoring layer initialization: Forgetting to call super().__init__() can break the model.
• Overfitting: Layers too complex cause training accuracy to be high but validation accuracy low.
• Data leakage: If layers or data preprocessing leak test info, metrics look falsely good.
• Accuracy paradox: High accuracy can be misleading if classes are imbalanced.

Self-check question

Your model has 98% accuracy but only 12% recall on fraud cases. Is it good for production? Why or why not?

Answer: No, it is not good. The low recall means the model misses most fraud cases, which is dangerous. Even with high accuracy, missing fraud is costly. You need to improve recall, possibly by adjusting layers or training.

Key Result

Proper layer initialization in __init__ is key to enabling good training loss decrease and balanced accuracy, precision, and recall.

Practice

1. What is the main purpose of the __init__ method in a PyTorch layer class?

easy

A. To save the model to disk

B. To set up the parts of the layer like linear or dropout layers

C. To train the model on data

D. To run the forward pass of the model

Solution

1. Step 1: Understand the role of __init__ in PyTorch

   The __init__ method is used to prepare the layer by defining its parts, such as linear or dropout layers.

2. Step 2: Differentiate from other methods

   The forward pass runs the data through the layer, training updates weights, and saving stores the model. These are not done in __init__.

3. Final Answer:

   To set up the parts of the layer like linear or dropout layers -> Option B

4. Quick Check:

   __init__ prepares layer parts = D [OK]

Hint: Remember: __init__ builds the layer parts first [OK]

Common Mistakes:

• Confusing __init__ with the forward method
• Thinking __init__ trains the model
• Assuming __init__ saves the model

2. Which of the following is the correct way to start the __init__ method in a PyTorch layer class?

easy

A. def __init__(self): super()

B. def __init__(): super().__init__()

C. def __init__(self): super().__init__()

D. def __init__(self): init.super()

Solution

1. Step 1: Check the method signature

   The __init__ method must have self as the first parameter.

2. Step 2: Call the parent class initializer correctly

   Use super().__init__() to properly initialize the parent class.

3. Final Answer:

   def __init__(self): super().__init__() -> Option C

4. Quick Check:

   Correct __init__ start = B [OK]

Hint: Always include self and call super().__init__() first [OK]

Common Mistakes:

• Omitting self parameter
• Calling super() without __init__()
• Wrong method signature without self

3. What will be the output shape of the linear layer defined below inside __init__ when input tensor has shape (10, 5)?

class MyLayer(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = torch.nn.Linear(5, 3)

    def forward(self, x):
        return self.linear(x)

layer = MyLayer()
input_tensor = torch.randn(10, 5)
output = layer(input_tensor)
print(output.shape)

medium

A. torch.Size([10, 3])

B. torch.Size([5, 3])

C. torch.Size([10, 5])

D. torch.Size([3, 10])

Solution

1. Step 1: Understand the linear layer dimensions

   The linear layer is defined as Linear(5, 3), so it expects input features of size 5 and outputs size 3.

2. Step 2: Match input tensor shape and output shape

   The input tensor has shape (10, 5), meaning 10 samples with 5 features each. The output will have shape (10, 3) after the linear layer.

3. Final Answer:

   torch.Size([10, 3]) -> Option A

4. Quick Check:

   Input features 5 to output 3, batch 10 = torch.Size([10, 3]) [OK]

Hint: Output shape = (batch size, output features) [OK]

Common Mistakes:

• Confusing input and output feature sizes
• Mixing batch size with feature dimension
• Ignoring the linear layer's output size

4. Identify the error in the following __init__ method of a PyTorch layer:

class MyLayer(torch.nn.Module):
    def __init__(self):
        self.linear = torch.nn.Linear(4, 2)
        super().__init__()

medium

A. Missing return statement in __init__

B. self.linear should be defined outside __init__

C. No error, code is correct

D. super().__init__() must be called before defining layers

Solution

1. Step 1: Check order of super().__init__() call

   In PyTorch, super().__init__() must be called first inside __init__ to properly initialize the parent class.

2. Step 2: Identify the error in the code

   The code calls super().__init__() after defining self.linear, which is incorrect and can cause errors.

3. Final Answer:

   super().__init__() must be called before defining layers -> Option D

4. Quick Check:

   Call super().__init__() first in __init__ = C [OK]

Hint: Always call super().__init__() before defining layers [OK]

Common Mistakes:

• Defining layers before calling super().__init__()
• Thinking __init__ needs a return statement
• Defining layers outside __init__

5. You want to create a custom PyTorch layer that has two linear layers and a dropout layer. Which __init__ method correctly sets up these parts?

hard

A. def __init__(self):
super().__init__()
self.linear1 = torch.nn.Linear(10, 20)
self.linear2 = torch.nn.Linear(20, 5)
self.dropout = torch.nn.Dropout(0.3)

B. def __init__(self):
self.linear1 = torch.nn.Linear(10, 20)
self.linear2 = torch.nn.Linear(20, 5)
self.dropout = torch.nn.Dropout(0.3)
super().__init__()

C. def __init__(self):
super().__init__()
self.linear1 = torch.nn.Linear(20, 10)
self.linear2 = torch.nn.Linear(5, 20)
self.dropout = torch.nn.Dropout(0.3)

D. def __init__(self):
super().__init__()
self.linear1 = torch.nn.Linear(10, 20)
self.linear2 = torch.nn.Linear(5, 20)
self.dropout = torch.nn.Dropout(0.3)

Solution

1. Step 1: Check the order of super().__init__() call

   It must be called first to initialize the parent class properly.

2. Step 2: Verify layer dimensions and definitions

   Linear layers should connect properly: first from 10 to 20, second from 20 to 5. Dropout is correctly defined with 0.3 probability.

3. Step 3: Identify the correct option

   def __init__(self): super().__init__() self.linear1 = torch.nn.Linear(10, 20) self.linear2 = torch.nn.Linear(20, 5) self.dropout = torch.nn.Dropout(0.3) matches correct order, layer sizes, and dropout setup. Codes that call super().__init__() last or have mismatched dimensions like Linear(20,10) and Linear(5,20) are incorrect.

4. Final Answer:

   def __init__(self): super().__init__() self.linear1 = torch.nn.Linear(10, 20) self.linear2 = torch.nn.Linear(20, 5) self.dropout = torch.nn.Dropout(0.3) -> Option A

5. Quick Check:

   Correct order and layer sizes = A [OK]

Hint: Call super first, then define layers with matching sizes [OK]

Common Mistakes:

• Calling super().__init__() last
• Using wrong input/output sizes for linear layers
• Misplacing dropout layer definition