Residual analysis in ML Python - Model Pipeline Trace

Model Pipeline - Residual analysis

Residual analysis helps us check how well a machine learning model fits the data by looking at the differences between actual and predicted values. It shows if the model makes consistent errors or if there are patterns it misses.

Data Flow - 6 Stages

1Data input

1000 rows x 2 columns→Load dataset with features and target values→1000 rows x 2 columns

Feature: house size, Target: house price

↓

2Train/test split

1000 rows x 2 columns→Split data into training and testing sets→800 rows x 2 columns (train), 200 rows x 2 columns (test)

Training: 800 houses, Testing: 200 houses

↓

3Model training

800 rows x 1 feature→Train linear regression model on training data→Trained model

Model learns relationship between house size and price

↓

4Prediction on test set

200 rows x 1 feature→Model predicts house prices for test data→200 predicted values

Predicted prices for 200 houses

↓

5Calculate residuals

200 actual values, 200 predicted values→Subtract predicted values from actual values→200 residual values

Residual = actual price - predicted price

↓

6Residual analysis

200 residual values→Plot residuals and check for patterns or randomness→Residual plot and summary statistics

Check if residuals are randomly scattered around zero

Training Trace - Epoch by Epoch

Loss
15000 | *
13000 |  *
11000 |   *
 9000 |    *
 7000 |     *
 5000 |      
       ----------------
        1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	15000.0	N/A	Initial loss is high as model starts learning
2	12000.0	N/A	Loss decreases as model improves
3	9000.0	N/A	Loss continues to decrease steadily
4	7000.0	N/A	Model fits data better, loss lowers
5	6000.0	N/A	Loss stabilizes, model converges

Prediction Trace - 4 Layers

Layer 1: Input feature

Layer 2: Linear regression prediction

Layer 3: Calculate residual

Layer 4: Residual analysis

Model Quiz - 3 Questions

What does a residual represent in residual analysis?

A Sum of actual and predicted values

B Difference between actual and predicted values

C Predicted value only

D Actual value only

Key Insight

Residual analysis is a simple but powerful way to check if a model predicts well. By looking at residuals, we can find if the model misses patterns or if errors are random, helping us improve or trust the model.

Practice

1. What does a residual represent in machine learning?

easy

A. The difference between the actual and predicted values

B. The predicted value itself

C. The average of all predicted values

D. The sum of all actual values

Solution

1. Step 1: Understand residual definition

   A residual is the error between what the model predicts and what actually happened.

2. Step 2: Match with options

   The difference between the actual and predicted values correctly states this difference, while others describe unrelated concepts.

3. Final Answer:

   The difference between the actual and predicted values -> Option A

4. Quick Check:

   Residual = Actual - Predicted [OK]

Hint: Residuals = Actual minus Predicted values [OK]

Common Mistakes:

• Confusing residual with predicted value
• Thinking residual is the average error
• Mixing residual with total sum of values

2. Which of the following is the correct formula to calculate a residual for a data point?

easy

A. Residual = Actual value + Predicted value

B. Residual = Predicted value - Actual value

C. Residual = Actual value - Predicted value

D. Residual = Predicted value / Actual value

Solution

1. Step 1: Recall residual calculation formula

   Residual is calculated as Actual value minus Predicted value.

2. Step 2: Identify correct option

   Residual = Actual value - Predicted value matches the formula exactly; others are incorrect operations.

3. Final Answer:

   Residual = Actual value - Predicted value -> Option C

4. Quick Check:

   Residual = Actual - Predicted [OK]

Hint: Subtract predicted from actual to get residual [OK]

Common Mistakes:

• Reversing the subtraction order
• Adding values instead of subtracting
• Dividing values instead of subtracting

3. Given the actual values [3, 5, 7] and predicted values [2, 5, 8], what is the residual for the third data point?

medium

A. 1

B. -1

C. 0

D. 2

Solution

1. Step 1: Identify actual and predicted values for third point

   Actual = 7, Predicted = 8 for the third data point.

2. Step 2: Calculate residual

   Residual = Actual - Predicted = 7 - 8 = -1.

3. Final Answer:

   -1 -> Option B

4. Quick Check:

   7 - 8 = -1 [OK]

Hint: Subtract predicted from actual for the point [OK]

Common Mistakes:

• Mixing up actual and predicted values
• Calculating predicted minus actual
• Using wrong index for data point

4. You have a residuals list: [2, -3, 4, -1]. Which code snippet correctly calculates the mean squared error (MSE)?

medium

A. mse = sum([r**2 for r in residuals]) / len(residuals)

B. mse = sum([abs(r) for r in residuals]) / len(residuals)

C. mse = max(residuals) - min(residuals)

D. mse = sum(residuals) / len(residuals)

Solution

1. Step 1: Understand MSE formula

   MSE is the average of squared residuals, so square each residual and average them.

2. Step 2: Match code to formula

   mse = sum([r**2 for r in residuals]) / len(residuals) squares each residual and divides by count, matching MSE calculation.

3. Final Answer:

   mse = sum([r**2 for r in residuals]) / len(residuals) -> Option A

4. Quick Check:

   MSE = average of squared residuals [OK]

Hint: Square residuals then average for MSE [OK]

Common Mistakes:

• Using sum instead of sum of squares
• Calculating mean absolute error instead
• Using max-min difference instead of average

5. A model's residual plot shows a clear curve pattern instead of random scatter. What does this indicate about the model?

hard

A. The model fits the data perfectly

B. The model has no bias

C. The residuals are normally distributed

D. The model misses a non-linear pattern in data

Solution

1. Step 1: Interpret residual plot pattern

   A curve pattern in residuals suggests systematic error, meaning the model misses some pattern.

2. Step 2: Identify what pattern means

   This usually means the model is too simple and misses non-linear relationships.

3. Final Answer:

   The model misses a non-linear pattern in data -> Option D

4. Quick Check:

   Curved residuals = missed non-linear pattern [OK]

Hint: Curved residuals mean model misses pattern [OK]

Common Mistakes:

• Assuming perfect fit from residual pattern
• Confusing residual distribution with residual pattern
• Ignoring bias presence in model