Residual analysis in ML Python - ML Experiment: Train & Evaluate

Experiment - Residual analysis

Problem:You have trained a simple linear regression model to predict house prices based on size. The model shows decent accuracy, but you want to check if the errors (residuals) are random or show patterns.

Current Metrics:Mean Squared Error (MSE): 2500000, R-squared: 0.75

Issue:The model might have systematic errors that are not visible from accuracy metrics alone. Residuals might show patterns indicating model problems.

Your Task

Perform residual analysis by plotting residuals and interpreting if errors are random or patterned. Identify if model assumptions hold.

Use the existing trained linear regression model

Do not change the model architecture or retrain

Use Python libraries matplotlib and scikit-learn

Hint 1

Hint 2

Hint 3

Hint 4

Solution

ML Python

import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score

# Sample data: house sizes (in 1000 sq ft) and prices (in $1000)
X = np.array([[1.0], [1.5], [2.0], [2.5], [3.0], [3.5], [4.0], [4.5], [5.0], [5.5]])
y = np.array([150, 200, 250, 300, 350, 400, 450, 500, 550, 600])

# Train linear regression model
model = LinearRegression()
model.fit(X, y)

# Predict prices
y_pred = model.predict(X)

# Calculate residuals
residuals = y - y_pred

# Print metrics
mse = mean_squared_error(y, y_pred)
r2 = r2_score(y, y_pred)
print(f"Mean Squared Error: {mse:.2f}")
print(f"R-squared: {r2:.2f}")

# Plot residuals vs predicted values
plt.scatter(y_pred, residuals, color='blue')
plt.axhline(y=0, color='red', linestyle='--')
plt.xlabel('Predicted Prices')
plt.ylabel('Residuals (Actual - Predicted)')
plt.title('Residual Plot')
plt.show()

Added calculation of residuals as difference between actual and predicted values

Created a scatter plot of residuals against predicted values

Added a horizontal line at zero to help visualize residual distribution

Results Interpretation

Before Residual Analysis: MSE = 2500000, R-squared = 0.75 (model seemed okay)

After Residual Analysis: MSE = 0.00, R-squared = 1.00 (perfect fit on sample data)

The residual plot shows points scattered evenly around zero with no clear pattern, indicating errors are random and model assumptions hold.

Residual analysis helps check if a model's errors are random or patterned. Random residuals mean the model fits well and assumptions are valid. Patterns suggest problems like missing variables or wrong model form.

Bonus Experiment

Try residual analysis on a dataset where the relationship is not linear, for example, house prices with a quadratic relation to size.

💡 Hint

Generate data with a squared term and fit a linear model. Residuals will show a pattern indicating the linear model is not enough.

Practice

1. What does a residual represent in machine learning?

easy

A. The difference between the actual and predicted values

B. The predicted value itself

C. The average of all predicted values

D. The sum of all actual values

Solution

1. Step 1: Understand residual definition

   A residual is the error between what the model predicts and what actually happened.

2. Step 2: Match with options

   The difference between the actual and predicted values correctly states this difference, while others describe unrelated concepts.

3. Final Answer:

   The difference between the actual and predicted values -> Option A

4. Quick Check:

   Residual = Actual - Predicted [OK]

Hint: Residuals = Actual minus Predicted values [OK]

Common Mistakes:

• Confusing residual with predicted value
• Thinking residual is the average error
• Mixing residual with total sum of values

2. Which of the following is the correct formula to calculate a residual for a data point?

easy

A. Residual = Actual value + Predicted value

B. Residual = Predicted value - Actual value

C. Residual = Actual value - Predicted value

D. Residual = Predicted value / Actual value

Solution

1. Step 1: Recall residual calculation formula

   Residual is calculated as Actual value minus Predicted value.

2. Step 2: Identify correct option

   Residual = Actual value - Predicted value matches the formula exactly; others are incorrect operations.

3. Final Answer:

   Residual = Actual value - Predicted value -> Option C

4. Quick Check:

   Residual = Actual - Predicted [OK]

Hint: Subtract predicted from actual to get residual [OK]

Common Mistakes:

• Reversing the subtraction order
• Adding values instead of subtracting
• Dividing values instead of subtracting

3. Given the actual values [3, 5, 7] and predicted values [2, 5, 8], what is the residual for the third data point?

medium

A. 1

B. -1

C. 0

D. 2

Solution

1. Step 1: Identify actual and predicted values for third point

   Actual = 7, Predicted = 8 for the third data point.

2. Step 2: Calculate residual

   Residual = Actual - Predicted = 7 - 8 = -1.

3. Final Answer:

   -1 -> Option B

4. Quick Check:

   7 - 8 = -1 [OK]

Hint: Subtract predicted from actual for the point [OK]

Common Mistakes:

• Mixing up actual and predicted values
• Calculating predicted minus actual
• Using wrong index for data point

4. You have a residuals list: [2, -3, 4, -1]. Which code snippet correctly calculates the mean squared error (MSE)?

medium

A. mse = sum([r**2 for r in residuals]) / len(residuals)

B. mse = sum([abs(r) for r in residuals]) / len(residuals)

C. mse = max(residuals) - min(residuals)

D. mse = sum(residuals) / len(residuals)

Solution

1. Step 1: Understand MSE formula

   MSE is the average of squared residuals, so square each residual and average them.

2. Step 2: Match code to formula

   mse = sum([r**2 for r in residuals]) / len(residuals) squares each residual and divides by count, matching MSE calculation.

3. Final Answer:

   mse = sum([r**2 for r in residuals]) / len(residuals) -> Option A

4. Quick Check:

   MSE = average of squared residuals [OK]

Hint: Square residuals then average for MSE [OK]

Common Mistakes:

• Using sum instead of sum of squares
• Calculating mean absolute error instead
• Using max-min difference instead of average

5. A model's residual plot shows a clear curve pattern instead of random scatter. What does this indicate about the model?

hard

A. The model fits the data perfectly

B. The model has no bias

C. The residuals are normally distributed

D. The model misses a non-linear pattern in data

Solution

1. Step 1: Interpret residual plot pattern

   A curve pattern in residuals suggests systematic error, meaning the model misses some pattern.

2. Step 2: Identify what pattern means

   This usually means the model is too simple and misses non-linear relationships.

3. Final Answer:

   The model misses a non-linear pattern in data -> Option D

4. Quick Check:

   Curved residuals = missed non-linear pattern [OK]

Hint: Curved residuals mean model misses pattern [OK]

Common Mistakes:

• Assuming perfect fit from residual pattern
• Confusing residual distribution with residual pattern
• Ignoring bias presence in model