First ML prediction (linear regression) in ML Python - Model Metrics & Evaluation

Metrics & Evaluation - First ML prediction (linear regression)

Which metric matters for this concept and WHY

For linear regression, we want to see how close our predicted numbers are to the real numbers. The best metrics are Mean Squared Error (MSE) and Root Mean Squared Error (RMSE). They measure the average size of the mistakes our model makes. Smaller values mean better predictions. We also use R-squared to know how much of the data's variation our model explains. A higher R-squared (close to 1) means the model fits well.

Confusion matrix or equivalent visualization (ASCII)

Linear regression predicts continuous values, so confusion matrix (used for categories) does not apply here. Instead, we look at error values like this example:

    Actual values:    [3, 5, 7, 9]
    Predicted values: [2.8, 5.1, 6.9, 9.2]
    Errors:          [-0.2, 0.1, -0.1, 0.2]
    Squared errors:  [0.04, 0.01, 0.01, 0.04]
    MSE = (0.04+0.01+0.01+0.04)/4 = 0.025
    RMSE = sqrt(0.025) = 0.158
    

Precision vs Recall (or equivalent tradeoff) with concrete examples

Precision and recall are for classification, not regression. For regression, the tradeoff is between bias and variance. A simple model may have high bias (wrong on average) but low variance (stable). A complex model may fit training data perfectly but fail on new data (high variance). We want a balance so errors are low and predictions generalize well.

Example: Predicting house prices. If the model always predicts the average price, it has high bias (misses details). If it fits every training house perfectly but fails on new houses, it has high variance.

What "good" vs "bad" metric values look like for this use case

Good values:

• MSE and RMSE close to 0 (small errors)
• R-squared close to 1 (explains most variation)

Bad values:

• High MSE or RMSE (large average errors)
• R-squared near 0 or negative (model does not explain data well)

For example, if RMSE is 10, but house prices range from 100 to 200, errors are large. If RMSE is 1, errors are small.

Metrics pitfalls (accuracy paradox, data leakage, overfitting indicators)

• Using accuracy: Accuracy is for classification, not regression. Using it here is wrong.
• Data leakage: If test data leaks into training, errors look too small and model seems better than it is.
• Overfitting: Very low training error but high test error means model memorizes training data and won't predict well on new data.
• Ignoring scale: MSE depends on the scale of the target variable. Comparing MSE across different problems without scaling is misleading.

Self-check: Your model has RMSE = 15 but house prices range from 100 to 200. Is it good?

No, RMSE of 15 means on average the model is off by 15 units, which is large compared to the price range (100-200). This means predictions are not very accurate and the model needs improvement.

Key Result

For linear regression, low MSE/RMSE and high R-squared show good prediction accuracy.

Practice

1. What is the main goal of linear regression in machine learning?

easy

A. To classify data into categories

B. To find a line that best fits the data points

C. To cluster data points into groups

D. To reduce the number of features

Solution

1. Step 1: Understand linear regression purpose

   Linear regression aims to find a straight line that best fits the relationship between input and output data.

2. Step 2: Compare with other options

   Classification, clustering, and feature reduction are different tasks, not the goal of linear regression.

3. Final Answer:

   To find a line that best fits the data points -> Option B

4. Quick Check:

   Linear regression = line fitting [OK]

Hint: Linear regression fits a line, not groups or categories [OK]

Common Mistakes:

• Confusing regression with classification
• Thinking linear regression clusters data
• Believing it reduces features

2. Which of the following is the correct Python syntax to create a simple linear regression model using scikit-learn?

easy

A. model = LinearRegression()

B. model = LinearRegression.fit()

C. model = LinearRegression.fit(X, y)

D. model = LinearRegression(X, y)

Solution

1. Step 1: Recall model creation syntax

   In scikit-learn, you first create the model object with LinearRegression(), without arguments.

2. Step 2: Understand fitting method

   The fit method is called on the model object with data, e.g., model.fit(X, y), not during creation.

3. Final Answer:

   model = LinearRegression() -> Option A

4. Quick Check:

   Model creation uses class name with parentheses [OK]

Hint: Create model with class(), fit separately [OK]

Common Mistakes:

• Calling fit during model creation
• Passing data to constructor
• Using fit as a class method

3. Given the code below, what will be the predicted value for X_new = [[5]]?

from sklearn.linear_model import LinearRegression
X = [[1], [2], [3], [4]]
y = [2, 4, 6, 8]
model = LinearRegression()
model.fit(X, y)
prediction = model.predict([[5]])
print(prediction)

medium

A. [9.]

B. [12.]

C. [8.]

D. [10.]

Solution

1. Step 1: Understand the training data pattern

   The output y doubles the input X: y = 2 * X.

2. Step 2: Predict for X_new = 5

   Using the pattern, prediction = 2 * 5 = 10.

3. Final Answer:

   [10.] -> Option D

4. Quick Check:

   2 x 5 = 10 [OK]

Hint: Check pattern in training data, apply to new input [OK]

Common Mistakes:

• Ignoring the pattern and guessing
• Using last y value as prediction
• Confusing input and output

4. What is wrong with the following code snippet for training a linear regression model?

from sklearn.linear_model import LinearRegression
X = [1, 2, 3, 4]
y = [2, 4, 6, 8]
model = LinearRegression()
model.fit(X, y)

medium

A. X should be a 2D array, not a 1D list

B. y should be a 2D array, not a list

C. LinearRegression() requires parameters in constructor

D. fit() method should be called with no arguments

Solution

1. Step 1: Check input shape requirements

   scikit-learn expects X as a 2D array (list of lists), but X is a 1D list here.

2. Step 2: Confirm y shape and method usage

   y can be 1D list, constructor needs no parameters, fit requires X and y arguments.

3. Final Answer:

   X should be a 2D array, not a 1D list -> Option A

4. Quick Check:

   X must be 2D for fit() [OK]

Hint: X must be 2D array for sklearn fit() [OK]

Common Mistakes:

• Passing 1D list for X
• Thinking y must be 2D
• Adding parameters to LinearRegression()

5. You trained a linear regression model on data where y = 3x + 2. After training, you want to predict the output for x = 10. Which code snippet correctly predicts and prints the result?

hard

A. model.predict([10])\nprint(prediction)

B. prediction = model.predict([10, 2])\nprint(prediction)

C. prediction = model.predict([[10]])\nprint(prediction[0])

D. prediction = model.predict(10)\nprint(prediction)

Solution

1. Step 1: Understand input shape for prediction

   scikit-learn expects a 2D array for prediction input, so [[10]] is correct.

2. Step 2: Access prediction output correctly

   predict returns an array; to print the single value, use prediction[0].

3. Final Answer:

   prediction = model.predict([[10]]) print(prediction[0]) -> Option C

4. Quick Check:

   Input 2D array, print first prediction [OK]

Hint: Use 2D input for predict, print first element [OK]

Common Mistakes:

• Passing 1D list or scalar to predict
• Printing prediction array directly without indexing
• Passing wrong shaped input with extra values