Multiple linear regression in ML Python - Model Pipeline Trace

Model Pipeline - Multiple linear regression

This pipeline uses multiple linear regression to predict a number based on several input features. It learns how each feature affects the result by finding the best straight-line fit through the data.

Data Flow - 4 Stages

1Data input

1000 rows x 4 columns→Raw data with 3 features (e.g., size, bedrooms, age) and 1 target (price)→1000 rows x 4 columns

Row example: [1200 sqft, 3 bedrooms, 10 years, 350000]

↓

2Preprocessing

1000 rows x 4 columns→Normalize features to similar scale→1000 rows x 4 columns

Normalized example: [0.6, 0.75, 0.2, 0.7]

↓

3Feature matrix and target vector split

1000 rows x 4 columns→Separate features (X) and target (y)→X: 1000 rows x 3 columns, y: 1000 rows x 1 column

X example: [0.6, 0.75, 0.2], y example: [350000]

↓

4Model training

X: 1000 rows x 3 columns, y: 1000 rows x 1 column→Fit linear regression coefficients to minimize error→Model with 3 coefficients + 1 intercept

Coefficients: [250000, 40000, -15000], Intercept: 50000

Training Trace - Epoch by Epoch

Loss
|
|***************
|************
|********
|*****
|***
|__|__|__|__|__
  1  2  3  4  5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	12000000000	N/A	Initial large error before fitting
2	8000000000	N/A	Loss decreased as model starts fitting
3	5000000000	N/A	Loss continues to decrease steadily
4	3000000000	N/A	Model improving fit to data
5	1500000000	N/A	Loss much lower, model converging

Prediction Trace - 3 Layers

Layer 1: Input features

Layer 2: Multiply features by coefficients

Layer 3: Sum weighted features and add intercept

Model Quiz - 3 Questions

What does the model learn during training?

A The best coefficients to multiply each feature

B How to classify data into categories

C How to cluster similar data points

D How to generate new data samples

Key Insight

Multiple linear regression finds the best straight-line fit by learning coefficients for each feature. Normalizing features helps the model learn efficiently. Loss decreases as the model improves its predictions.

Practice

1. What does multiple linear regression do in simple terms?

easy

A. It predicts a result using several input features combined.

B. It groups data into clusters based on similarity.

C. It reduces the number of features by combining them.

D. It finds the shortest path between two points.

Solution

1. Step 1: Understand the purpose of multiple linear regression

   Multiple linear regression uses multiple inputs to predict a single output by finding the best weights for each input.

2. Step 2: Compare options with this understanding

   Only It predicts a result using several input features combined. correctly describes predicting a result using several inputs combined.

3. Final Answer:

   It predicts a result using several input features combined. -> Option A

4. Quick Check:

   Multiple inputs predict output = A [OK]

Hint: Remember: multiple inputs, one output prediction [OK]

Common Mistakes:

• Confusing regression with clustering
• Thinking it reduces features instead of weighting them
• Mixing it up with pathfinding algorithms

2. Which of the following is the correct formula for multiple linear regression prediction?

easy

A. y = w0 + w1*x1 + w2*x2 + ... + wn*xn

B. y = w0 * x1 + w1 * x2 + ... + wn * xn

C. y = w0 / x1 + w1 / x2 + ... + wn / xn

D. y = w0 - w1*x1 - w2*x2 - ... - wn*xn

Solution

1. Step 1: Recall the multiple linear regression formula

   The prediction is a sum of a bias term plus each input multiplied by its weight: y = w0 + w1*x1 + w2*x2 + ... + wn*xn.

2. Step 2: Check each option against the formula

   Only y = w0 + w1*x1 + w2*x2 + ... + wn*xn matches the correct formula with addition and multiplication.

3. Final Answer:

   y = w0 + w1*x1 + w2*x2 + ... + wn*xn -> Option A

4. Quick Check:

   Sum of weighted inputs plus bias = B [OK]

Hint: Sum weighted inputs plus bias for prediction [OK]

Common Mistakes:

• Using multiplication instead of addition between terms
• Dividing weights by inputs
• Subtracting weights instead of adding

3. Given weights w0=2, w1=3, w2=-1 and inputs x1=4, x2=5, what is the predicted value y using multiple linear regression?

medium

A. 9.5

B. 9

C. 5

D. 10

Solution

1. Step 1: Apply the formula y = w0 + w1*x1 + w2*x2

   Substitute values: y = 2 + 3*4 + (-1)*5.

2. Step 2: Calculate the prediction

   Calculate: 3*4 = 12, (-1)*5 = -5, sum = 2 + 12 - 5 = 9.

3. Step 3: Recheck calculation

   Sum is 9, but 10 says 10, so re-check: 2 + 12 - 5 = 9, so correct answer is 9.

4. Final Answer:

   9 -> Option B

5. Quick Check:

   2 + 12 - 5 = 9 [OK]

Hint: Multiply weights by inputs, then add bias [OK]

Common Mistakes:

• Adding weights instead of multiplying by inputs
• Forgetting to add bias term w0
• Mixing up signs of weights

4. Identify the error in this Python code for multiple linear regression prediction:

def predict(x1, x2, w0, w1, w2):
    return w0 + w1 * x1 + w2 + x2

print(predict(3, 4, 1, 2, 3))

medium

A. The print statement syntax is incorrect.

B. The function is missing a return statement.

C. The function parameters are in wrong order.

D. The term w2 + x2 should be w2 * x2 for correct multiplication.

Solution

1. Step 1: Analyze the formula inside the function

   The formula should multiply each input by its weight: w0 + w1*x1 + w2*x2.

2. Step 2: Identify the mistake in the code

   The code uses w2 + x2 instead of w2 * x2, which adds instead of multiplies.

3. Final Answer:

   The term w2 + x2 should be w2 * x2 for correct multiplication. -> Option D

4. Quick Check:

   Multiplication needed, not addition = A [OK]

Hint: Check all input-weight pairs use multiplication [OK]

Common Mistakes:

• Using + instead of * between weight and input
• Forgetting return statement (not here though)
• Mixing parameter order
• Incorrect print syntax

5. You have a dataset with features: hours studied (x1), hours slept (x2), and exam score (y). You fit a multiple linear regression model and get weights: w0=5, w1=4, w2=2. What does the weight w1=4 tell you about hours studied?

hard

A. Hours slept affects exam score more than hours studied.

B. Hours studied has no effect on exam score.

C. For each extra hour studied, exam score increases by 4 points, holding sleep constant.

D. Exam score decreases by 4 points for each hour studied.

Solution

1. Step 1: Understand the meaning of weights in multiple linear regression

   Each weight shows how much the output changes when that input increases by one unit, keeping others fixed.

2. Step 2: Interpret w1=4 for hours studied

   A weight of 4 means exam score increases by 4 points for each additional hour studied, assuming hours slept stays the same.

3. Final Answer:

   For each extra hour studied, exam score increases by 4 points, holding sleep constant. -> Option C

4. Quick Check:

   Weight shows effect per unit input = D [OK]

Hint: Weight = change in output per unit input [OK]

Common Mistakes:

• Thinking weight means no effect
• Confusing positive and negative effect
• Mixing effects of different features