Sequential model shortcut in PyTorch - Model Metrics & Evaluation

Metrics & Evaluation - Sequential model shortcut

Which metric matters for Sequential model shortcut and WHY

When using a sequential model shortcut, like a skip connection in a neural network, the key metrics to watch are training loss and validation accuracy. These show if the shortcut helps the model learn better and generalize well.

Because shortcuts help information flow, they often reduce vanishing gradients, so the model trains faster and achieves better accuracy. Watching loss and accuracy helps confirm this improvement.

Confusion matrix example

For classification tasks using a sequential model shortcut, the confusion matrix shows how well the model predicts each class.

      Actual \ Predicted |  Positive | Negative
      -------------------|-----------|---------
      Positive           |    85     |   15
      Negative           |    10     |   90
    

Here, True Positives (TP) = 85, False Negatives (FN) = 15, False Positives (FP) = 10, True Negatives (TN) = 90.

Precision vs Recall tradeoff with examples

Using shortcuts can improve both precision and recall by helping the model learn better features.

Precision measures how many predicted positives are actually positive. For example, in spam detection, high precision means few good emails are marked as spam.

Recall measures how many actual positives are found. For example, in disease detection, high recall means fewer sick patients are missed.

Shortcuts help balance this tradeoff by improving learning, but depending on the task, you might prioritize one metric over the other.

Good vs Bad metric values for Sequential model shortcut

Good: Validation accuracy above 85%, loss steadily decreasing, precision and recall both above 80%. This means the shortcut helps the model learn and generalize well.

Bad: Validation accuracy stuck near chance (e.g., 50% for binary), loss not improving or increasing, precision or recall very low (below 50%). This suggests the shortcut is not helping or model is overfitting.

Common pitfalls with metrics

• Accuracy paradox: High accuracy can be misleading if classes are imbalanced. Always check precision and recall.
• Data leakage: If test data leaks into training, metrics look unrealistically good.
• Overfitting: Training accuracy very high but validation accuracy low means model memorizes training data, shortcut may not help.
• Ignoring loss curves: Only looking at accuracy can miss if model is unstable or not converging.

Self-check question

Your model with a sequential shortcut has 98% accuracy but only 12% recall on fraud cases. Is it good for production?

Answer: No. Despite high accuracy, the model misses most fraud cases (low recall). For fraud detection, recall is critical to catch fraud. This model needs improvement.

Key Result

Sequential model shortcuts improve training by reducing loss and increasing accuracy, but precision and recall must be checked to ensure balanced performance.

Practice

1. What is the main advantage of using torch.nn.Sequential in PyTorch?

easy

A. It allows stacking layers quickly without writing a forward method.

B. It automatically tunes hyperparameters during training.

C. It visualizes the model architecture graphically.

D. It replaces the need for a loss function.

Solution

1. Step 1: Understand what Sequential does

   torch.nn.Sequential stacks layers in order and handles the forward pass automatically.

2. Step 2: Compare other options

   Hyperparameter tuning, visualization, and replacing the loss function are not features of Sequential.

3. Final Answer:

   It allows stacking layers quickly without writing a forward method. -> Option A

4. Quick Check:

   Sequential = quick layer stacking [OK]

Hint: Sequential stacks layers; no manual forward needed [OK]

Common Mistakes:

• Thinking Sequential tunes hyperparameters
• Confusing Sequential with visualization tools
• Assuming Sequential replaces loss functions

2. Which of the following is the correct way to create a simple feedforward network with two linear layers using torch.nn.Sequential?

easy

A. model = nn.Sequential(nn.Linear(10, 5) nn.ReLU() nn.Linear(5, 2))

B. model = nn.Sequential([nn.Linear(10, 5), nn.ReLU(), nn.Linear(5, 2)])

C. model = nn.Sequential(nn.Linear(10, 5), nn.ReLU(), nn.Linear(5, 2))

D. model = nn.Sequential{nn.Linear(10, 5), nn.ReLU(), nn.Linear(5, 2)}

Solution

1. Step 1: Check correct syntax for Sequential

   nn.Sequential takes layers as separate arguments, separated by commas and enclosed in parentheses.

2. Step 2: Identify syntax errors in other options

   A misses commas between layers causing a syntax error, B uses a list which is invalid here, C uses curly braces which is wrong.

3. Final Answer:

   model = nn.Sequential(nn.Linear(10, 5), nn.ReLU(), nn.Linear(5, 2)) -> Option C

4. Quick Check:

   Sequential layers separated by commas in parentheses [OK]

Hint: Use commas between layers inside parentheses [OK]

Common Mistakes:

• Using square brackets inside Sequential
• Using curly braces instead of parentheses
• Forgetting commas between layers

3. Given the following code, what will be the output shape of output?

import torch
import torch.nn as nn
model = nn.Sequential(
    nn.Linear(4, 3),
    nn.ReLU(),
    nn.Linear(3, 2)
)
input = torch.randn(5, 4)
output = model(input)
print(output.shape)

medium

A. torch.Size([4, 2])

B. torch.Size([5, 3])

C. torch.Size([5, 4])

D. torch.Size([5, 2])

Solution

1. Step 1: Analyze input and first layer

   Input shape is (5,4). First Linear layer maps 4 features to 3, so output shape becomes (5,3).

2. Step 2: Analyze second Linear layer output

   Second Linear layer maps 3 features to 2, so final output shape is (5,2).

3. Final Answer:

   torch.Size([5, 2]) -> Option D

4. Quick Check:

   Input batch 5, final features 2 [OK]

Hint: Track input batch size and final layer output features [OK]

Common Mistakes:

• Confusing input and output feature sizes
• Ignoring batch dimension
• Assuming ReLU changes shape

4. What is wrong with this PyTorch Sequential model code?

import torch.nn as nn
model = nn.Sequential(
    nn.Linear(10, 5),
    nn.ReLU,
    nn.Linear(5, 2)
)

medium

A. Missing parentheses after nn.ReLU causing a type error.

B. Linear layers have wrong input-output sizes.

C. Sequential cannot contain activation functions.

D. Model should be defined using nn.Module subclass.

Solution

1. Step 1: Check usage of nn.ReLU

   nn.ReLU is a class; it must be called with parentheses to create an instance.

2. Step 2: Verify other options

   Linear layers sizes are compatible, Sequential supports activations, and subclassing is optional.

3. Final Answer:

   Missing parentheses after nn.ReLU causing a type error. -> Option A

4. Quick Check:

   Activation needs parentheses () [OK]

Hint: Always add () after activation layers [OK]

Common Mistakes:

• Forgetting () after activation functions
• Thinking Sequential disallows activations
• Believing subclassing is mandatory

5. You want to build a neural network with three layers: input size 8, hidden size 4, and output size 1. You also want to add dropout after the first layer and use ReLU activations. Which nn.Sequential model correctly implements this?

hard

A. nn.Sequential(nn.Linear(8,4), nn.ReLU, nn.Dropout(0.5), nn.Linear(4,1))

B. nn.Sequential(nn.Linear(8,4), nn.Dropout(0.5), nn.ReLU(), nn.Linear(4,1))

C. nn.Sequential(nn.Linear(8,4), nn.ReLU(), nn.Linear(4,1), nn.Dropout(0.5))

D. nn.Sequential(nn.Dropout(0.5), nn.Linear(8,4), nn.ReLU(), nn.Linear(4,1))

Solution

1. Step 1: Check layer order and activations

   Dropout should come after the first Linear layer and before ReLU to randomly drop activations during training.

2. Step 2: Verify correct usage of activations and dropout

   nn.Sequential(nn.Linear(8,4), nn.Dropout(0.5), nn.ReLU(), nn.Linear(4,1)) correctly uses nn.ReLU() with parentheses and places dropout before ReLU. Other options either miss parentheses or misplace dropout.

3. Final Answer:

   nn.Sequential(nn.Linear(8,4), nn.Dropout(0.5), nn.ReLU(), nn.Linear(4,1)) -> Option B

4. Quick Check:

   Dropout after Linear, before ReLU, with () [OK]

Hint: Dropout after Linear, before ReLU, use () for activations [OK]

Common Mistakes:

• Placing dropout after output layer
• Forgetting parentheses on ReLU
• Putting dropout before first Linear layer