Why regression predicts continuous values in ML Python - Why Metrics Matter

Metrics & Evaluation - Why regression predicts continuous values

Which metric matters for this concept and WHY

For regression, we predict continuous numbers like house prices or temperatures. So, metrics that measure how close our predictions are to the real numbers matter most. Common metrics are Mean Squared Error (MSE), Mean Absolute Error (MAE), and R-squared (R²). These tell us how far off our predictions are on average, helping us understand if the model is good at predicting continuous values.

Confusion matrix or equivalent visualization (ASCII)

Regression does not use a confusion matrix because it predicts numbers, not categories. Instead, we look at error values. Here is an example of prediction errors for 5 samples:

    Sample | Actual | Predicted | Error (Predicted - Actual)
    -------|--------|-----------|--------------------------
       1   |  3.5   |    3.7    |          0.2             
       2   |  7.0   |    6.5    |         -0.5             
       3   |  2.0   |    2.1    |          0.1             
       4   |  5.5   |    5.0    |         -0.5             
       5   |  4.0   |    4.2    |          0.2             
    

We use these errors to calculate metrics like MSE or MAE.

Precision vs Recall (or equivalent tradeoff) with concrete examples

In regression, instead of precision and recall, we balance bias and variance:

• Bias: How far off predictions are on average (underfitting if too high).
• Variance: How much predictions change with different data (overfitting if too high).

For example, if a model always predicts the average house price, it has high bias (not accurate). If it fits every tiny detail of training data, it has high variance (may fail on new data). Good regression models find a balance to predict continuous values well.

What "good" vs "bad" metric values look like for this use case

For regression:

• Good: Low MSE or MAE (close to 0), and R² close to 1 (explains most variation).
• Bad: High MSE or MAE (large errors), and R² close to 0 or negative (model predicts worse than average).

Example:

• MSE = 0.1 (good), MAE = 0.2 (good), R² = 0.95 (excellent)
• MSE = 10 (bad), MAE = 3 (bad), R² = 0.1 (poor)

Metrics pitfalls (accuracy paradox, data leakage, overfitting indicators)

• Using classification metrics: Confusion matrix or accuracy do not apply to regression.
• Ignoring scale: Large errors can be hidden if data values are very large; always check error scale.
• Data leakage: If test data leaks into training, error metrics look unrealistically good.
• Overfitting: Very low training error but high test error means model memorizes training data, not generalizing.

"Your model has 98% accuracy but 12% recall on fraud. Is it good?"

This question is for classification, not regression. For regression, imagine a model with very low error on training but high error on new data. That means it is not good because it does not predict continuous values well on unseen data. Always check error metrics on new data to know if the regression model is good.

Key Result

Regression models are best evaluated by error metrics like MSE and MAE that measure how close predicted continuous values are to actual values.

Practice

1. Why does regression predict continuous values instead of categories?

easy

A. Because it predicts yes or no answers

B. Because it groups data into fixed classes

C. Because it only works with text data

D. Because it estimates quantities that can take any value within a range

Solution

1. Step 1: Understand regression's purpose

   Regression is designed to predict numbers that can vary continuously, like prices or temperatures.

2. Step 2: Compare with classification

   Classification predicts fixed groups or categories, not continuous values.

3. Final Answer:

   Because it estimates quantities that can take any value within a range -> Option D

4. Quick Check:

   Regression predicts continuous values = Because it estimates quantities [OK]

Hint: Regression predicts numbers, classification predicts groups [OK]

Common Mistakes:

• Confusing regression with classification
• Thinking regression predicts categories
• Assuming regression only works with discrete values

2. Which of the following is the correct way to define a regression target variable in Python?

easy

A. target = [3.5, 7.2, 1.8]

B. target = ['cat', 'dog', 'bird']

C. target = ['red', 'blue', 'green']

D. target = ['yes', 'no', 'maybe']

Solution

1. Step 1: Identify regression target type

   Regression targets are continuous numbers, like 3.5 or 7.2.

2. Step 2: Check options for continuous values

   Only target = [3.5, 7.2, 1.8] contains numbers suitable for regression.

3. Final Answer:

   target = [3.5, 7.2, 1.8] -> Option A

4. Quick Check:

   Regression targets are numbers = target with floats [OK]

Hint: Regression targets are numeric lists, not categories [OK]

Common Mistakes:

• Using categorical strings as regression targets
• Confusing classification labels with regression targets
• Using boolean or text values for regression

3. You want to predict house prices using regression. Your dataset has features like size (sq ft), number of rooms, and age of the house. Why is regression the right choice here?

easy

A. Because house prices belong to fixed categories like cheap or expensive

B. Because house prices are continuous values that can vary widely

C. Because regression only works with text data

D. Because regression predicts yes/no answers

Solution

1. Step 1: Identify target variable type

   House prices are numbers that can take many values, not fixed groups.

2. Step 2: Match problem to regression

   Regression predicts continuous numeric values, perfect for price prediction.

3. Final Answer:

   Because house prices are continuous values that can vary widely -> Option B

4. Quick Check:

   Price prediction = continuous value regression [OK]

Hint: Predict continuous numbers like prices with regression [OK]

Common Mistakes:

• Treating prices as categories
• Choosing classification instead of regression
• Confusing regression with yes/no prediction

4. Given this simple regression model in Python using scikit-learn:

from sklearn.linear_model import LinearRegression
X = [[1], [2], [3], [4]]
y = [2.0, 4.1, 6.0, 8.1]
model = LinearRegression()
model.fit(X, y)
pred = model.predict([[5]])
print(round(pred[0], 1))

What will be the printed output?

medium

A. 10.1

B. 8.5

C. 9.9

D. 10.0

Solution

1. Step 1: Understand the data and model

   The data roughly follows y = 2 * x with small noise. The model fits a line close to y=2x.

2. Step 2: Predict for x=5

   Prediction is about 2 * 5 = 10, but slight noise shifts it to around 10.1.

3. Final Answer:

   10.1 -> Option A

4. Quick Check:

   Linear fit predicts ~10.1 for input 5 [OK]

Hint: Regression predicts near trend line values, not exact multiples [OK]

Common Mistakes:

• Assuming exact multiples without noise
• Confusing classification output with regression output
• Rounding incorrectly

5. You wrote this regression code but get an error:

from sklearn.linear_model import LinearRegression
X = [1, 2, 3, 4]
y = [2.0, 4.1, 6.0, 8.1]
model = LinearRegression()
model.fit(X, y)

What is the error and how to fix it?

medium

A. model.fit needs no arguments, remove X and y

B. y should be strings, fix with y = ['2.0', '4.1', '6.0', '8.1']

C. X should be a 2D array, fix with X = [[1], [2], [3], [4]]

D. LinearRegression only works with categorical data

Solution

1. Step 1: Identify input shape error

   scikit-learn expects X as 2D array (list of lists), but X is 1D list.

2. Step 2: Fix input shape

   Wrap each value in a list: X = [[1], [2], [3], [4]] to make it 2D.

3. Final Answer:

   X should be a 2D array, fix with X = [[1], [2], [3], [4]] -> Option C

4. Quick Check:

   Input X must be 2D array for regression [OK]

Hint: Regression input X must be 2D array, not 1D list [OK]

Common Mistakes:

• Passing 1D list instead of 2D array for X
• Changing y to strings incorrectly
• Removing required fit arguments