First PyTorch computation - Model Metrics & Evaluation

Metrics & Evaluation - First PyTorch computation

Which metric matters for this concept and WHY

When you do your first PyTorch computation, the main goal is to check if your code runs correctly and produces expected results. At this stage, metrics like loss and accuracy help you understand if your model learns well. Loss tells how far your model's predictions are from the true answers. Accuracy shows how many predictions are correct. These metrics matter because they guide you to improve your model step by step.

Confusion matrix or equivalent visualization (ASCII)

For classification tasks, a confusion matrix helps you see where your model makes mistakes. Here is an example for a simple two-class problem:

      Actual \ Predicted | Positive | Negative
      -------------------|----------|---------
      Positive           |    TP=8  |   FN=2  
      Negative           |    FP=1  |   TN=9  
    

This shows 8 true positives, 2 false negatives, 1 false positive, and 9 true negatives. You can calculate precision, recall, and accuracy from these numbers.

Precision vs Recall tradeoff with concrete examples

Imagine your first PyTorch model is for spam detection. Precision means how many emails marked as spam really are spam. Recall means how many actual spam emails your model finds.

If your model has high precision but low recall, it rarely marks good emails as spam but misses many spam emails. If it has high recall but low precision, it catches most spam but also marks many good emails as spam.

Depending on your goal, you choose which metric to improve. For your first computation, understanding this tradeoff helps you decide what to focus on next.

What "good" vs "bad" metric values look like for this use case

For your first PyTorch computation, good metrics mean your model learns something useful:

• Loss: Should decrease over time. If it stays the same or increases, your model might not be learning.
• Accuracy: Should improve from random guessing. For example, if you have 2 classes, random accuracy is about 50%. Getting 70% or more is good progress.

Bad metrics mean your model is not learning or your code has errors. For example, accuracy stuck at random level or loss not changing.

Metrics pitfalls (accuracy paradox, data leakage, overfitting indicators)

When starting with PyTorch, watch out for these common pitfalls:

• Accuracy paradox: High accuracy can be misleading if your data is unbalanced. For example, if 90% of data is one class, predicting that class always gives 90% accuracy but no real learning.
• Data leakage: Accidentally using test data during training can make metrics look perfect but the model will fail on new data.
• Overfitting: If training accuracy is very high but test accuracy is low, your model memorizes training data but does not generalize.

Self-check: Your model has 98% accuracy but 12% recall on fraud. Is it good?

No, this model is not good for fraud detection. Even though accuracy is high, recall is very low. This means the model misses most fraud cases (only finds 12%). In fraud detection, missing fraud is very costly, so recall is more important. You should improve recall even if accuracy drops a bit.

Key Result

Loss decreasing and accuracy improving from random guessing show your first PyTorch computation is working well.

Practice

1. What is a PyTorch tensor mainly used for?

easy

A. Holding numbers for math operations

B. Storing text data only

C. Creating web pages

D. Managing database connections

Solution

1. Step 1: Understand the role of tensors in PyTorch

   PyTorch tensors are like containers that hold numbers for math operations, similar to arrays.

2. Step 2: Compare options with tensor purpose

   Storing text data only, creating web pages, and managing database connections do not relate to math operations or number storage, so they are incorrect.

3. Final Answer:

   Holding numbers for math operations -> Option A

4. Quick Check:

   PyTorch tensor = numbers container [OK]

Hint: Tensors hold numbers for math, not text or web [OK]

Common Mistakes:

• Thinking tensors store only text
• Confusing tensors with web tools
• Assuming tensors manage databases

2. Which of the following is the correct way to create a tensor with values [1, 2, 3] in PyTorch?

easy

A. torch.tensor([1, 2, 3])

B. torch.array([1, 2, 3])

C. torch.list([1, 2, 3])

D. torch.vector([1, 2, 3])

Solution

1. Step 1: Recall PyTorch tensor creation syntax

   PyTorch uses torch.tensor() to create tensors from lists or arrays.

2. Step 2: Check each option's validity

   torch.vector([1, 2, 3]), torch.array([1, 2, 3]), and torch.list([1, 2, 3]) use incorrect functions that do not exist in PyTorch.

3. Final Answer:

   torch.tensor([1, 2, 3]) -> Option A

4. Quick Check:

   Use torch.tensor() to create tensors [OK]

Hint: Use torch.tensor() to create tensors from lists [OK]

Common Mistakes:

• Using torch.array which is not a PyTorch function
• Trying torch.list or torch.vector which don't exist
• Confusing PyTorch with NumPy syntax

3. What will be the output of this PyTorch code?

import torch
x = torch.tensor([2, 4])
y = torch.tensor([3, 1])
z = x + y
print(z)

medium

A. tensor([6, 4])

B. tensor([5, 5])

C. tensor([2, 4, 3, 1])

D. Error: cannot add tensors

Solution

1. Step 1: Add tensors element-wise

   Adding tensors x and y adds each element: 2+3=5 and 4+1=5.

2. Step 2: Confirm output format

   tensor([6, 4]) and tensor([2, 4, 3, 1]) show wrong sums or concatenation, and 'Error: cannot add tensors' is incorrect as addition is valid and produces tensor([5, 5]).

3. Final Answer:

   tensor([5, 5]) -> Option B

4. Quick Check:

   2+3=5 and 4+1=5 [OK]

Hint: Tensor addition is element-wise [OK]

Common Mistakes:

• Thinking addition concatenates tensors
• Mixing element-wise sum with multiplication
• Expecting an error for valid addition

4. Identify the error in this PyTorch code snippet:

import torch
x = torch.tensor([1, 2, 3])
y = torch.tensor([4, 5])
z = x + y
print(z)

medium

A. No error, code runs fine

B. Missing import statement

C. Incorrect tensor creation syntax

D. Tensors have different sizes, cannot add

Solution

1. Step 1: Check tensor sizes

   x has 3 elements, y has 2 elements; PyTorch cannot add tensors of different sizes element-wise.

2. Step 2: Identify error cause

   There is no missing import statement and tensor creation syntax is correct. 'No error, code runs fine' is wrong as code raises size mismatch error.

3. Final Answer:

   Tensors have different sizes, cannot add -> Option D

4. Quick Check:

   Tensor sizes must match for addition [OK]

Hint: Check tensor shapes before adding [OK]

Common Mistakes:

• Ignoring size mismatch error
• Assuming PyTorch auto-adjusts sizes
• Thinking import or syntax is wrong

5. You want to multiply two tensors element-wise and then find the sum of all elements in the result. Which PyTorch code correctly does this?

import torch
x = torch.tensor([1, 2, 3])
y = torch.tensor([4, 5, 6])
# Choose the correct next line

hard

A. result = torch.dot(x, y).sum()

B. result = x.matmul(y)

C. result = (x * y).sum()

D. result = x + y.sum()

Solution

1. Step 1: Understand element-wise multiplication and sum

   x * y multiplies each element: [1*4, 2*5, 3*6] = [4, 10, 18]. Then sum() adds all: 4+10+18=32.

2. Step 2: Evaluate options

   torch.dot(x, y).sum() applies dot product then sum again (dot product is not element-wise multiplication, and summing a scalar is redundant). x.matmul(y) does matrix multiplication, not element-wise. x + y.sum() adds x to sum of y, not multiplication.

3. Final Answer:

   result = (x * y).sum() -> Option C

4. Quick Check:

   Element-wise multiply then sum = 32 [OK]

Hint: Use (x * y).sum() for element-wise multiply then sum [OK]

Common Mistakes:

• Using matmul instead of element-wise multiply
• Applying sum twice incorrectly
• Adding tensors instead of multiplying