Polynomial regression pipeline in ML Python - Model Pipeline Trace

Model Pipeline - Polynomial regression pipeline

This pipeline shows how polynomial regression learns to fit curved data by adding extra features that are powers of the original input. It transforms data, trains a model, and improves predictions over time.

Data Flow - 5 Stages

1Raw Data Input

1000 rows x 1 column→Collect original single feature data points→1000 rows x 1 column

x = [1, 2, 3, 4, 5]

↓

2Polynomial Feature Expansion

1000 rows x 1 column→Create new features by raising input to powers 1 and 2 (x and x^2)→1000 rows x 2 columns

x = [1, 2, 3], transformed to [[1, 1], [2, 4], [3, 9]]

↓

3Train/Test Split

1000 rows x 2 columns→Split data into 800 training rows and 200 testing rows→Train: 800 rows x 2 columns, Test: 200 rows x 2 columns

Train features shape: (800, 2), Test features shape: (200, 2)

↓

4Model Training

Train: 800 rows x 2 columns→Fit linear regression model on polynomial features→Trained model with 2 coefficients plus intercept

Model learns weights for x and x^2 terms

↓

5Model Evaluation

Test: 200 rows x 2 columns→Predict and compare to true values to compute loss and R^2 score→Loss scalar and R^2 score scalar

Loss = 0.15, R^2 = 0.92

Training Trace - Epoch by Epoch

Loss
1.0 |*       
0.8 | *      
0.6 |  *     
0.4 |   *    
0.2 |    *   
0.0 +--------
      1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.85	0.45	Model starts with high loss and low accuracy
2	0.50	0.70	Loss decreases as model learns polynomial relationship
3	0.30	0.85	Model fits data better, accuracy improves
4	0.20	0.90	Loss continues to decrease, model converging
5	0.15	0.92	Final epoch shows good fit with low loss and high accuracy

Prediction Trace - 4 Layers

Layer 1: Input Feature

Layer 2: Polynomial Feature Expansion

Layer 3: Linear Model Prediction

Layer 4: Output Prediction

Model Quiz - 3 Questions

What does the polynomial feature expansion stage do?

A Calculates loss and accuracy

B Splits data into training and testing sets

C Adds new features by raising input to powers

D Predicts output values

Key Insight

Polynomial regression improves simple linear models by adding powers of input features, allowing the model to fit curved patterns in data. Training shows loss decreasing and accuracy increasing, confirming the model learns the relationship well.

Practice

1.

What is the main purpose of using polynomial regression instead of simple linear regression?

easy

A. To fit curved relationships between variables

B. To reduce the number of features

C. To speed up training time

D. To handle missing data automatically

Solution

1. Step 1: Understand linear regression limitation

   Linear regression fits straight lines, which cannot capture curves in data.

2. Step 2: Role of polynomial regression

   Polynomial regression fits curved lines by adding powers of features, capturing non-linear patterns.

3. Final Answer:

   To fit curved relationships between variables -> Option A

4. Quick Check:

   Polynomial regression = curved fit [OK]

Hint: Polynomial regression fits curves, not just straight lines [OK]

Common Mistakes:

• Thinking polynomial regression reduces features
• Assuming it speeds up training
• Believing it handles missing data automatically

2.

Which of the following is the correct way to create a polynomial regression pipeline in Python using sklearn?

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression

pipeline = Pipeline([
    ('poly', PolynomialFeatures(degree=2)),
    ('linear', LinearRegression())
])

easy

A. pipeline = Pipeline([('poly', PolynomialFeatures(degree=2)), ('linear', LinearRegression())])

B. pipeline = Pipeline([('linear', LinearRegression()), ('poly', PolynomialFeatures(degree=2))])

C. pipeline = Pipeline([('poly', LinearRegression()), ('linear', PolynomialFeatures(degree=2))])

D. pipeline = Pipeline([('poly', PolynomialFeatures()), ('linear', LinearRegression(degree=2))])

Solution

1. Step 1: Order of pipeline steps

   PolynomialFeatures must come before LinearRegression to transform data first.

2. Step 2: Correct usage of classes and parameters

   PolynomialFeatures takes degree parameter; LinearRegression does not take degree.

3. Final Answer:

   pipeline = Pipeline([('poly', PolynomialFeatures(degree=2)), ('linear', LinearRegression())]) -> Option A

4. Quick Check:

   PolynomialFeatures before LinearRegression [OK]

Hint: Put PolynomialFeatures before LinearRegression in pipeline [OK]

Common Mistakes:

• Swapping order of pipeline steps
• Passing degree to LinearRegression
• Omitting degree in PolynomialFeatures

3.

Given the following code, what will print(y_pred) output?

import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression

X = np.array([[1], [2], [3]])
y = np.array([1, 4, 9])

pipeline = Pipeline([
    ('poly', PolynomialFeatures(degree=2)),
    ('linear', LinearRegression())
])
pipeline.fit(X, y)
y_pred = pipeline.predict(np.array([[4]]))
print(np.round(y_pred, 2))

medium

A. [10.0]

B. [8.0]

C. [4.0]

D. [16.0]

Solution

1. Step 1: Understand data and model

   X = [[1],[2],[3]] with y = [1,4,9] fits y = x^2 perfectly.

2. Step 2: Predict for X=4 using polynomial degree 2

   Model learns y = x^2, so prediction at 4 is 4^2 = 16.

3. Final Answer:

   [16.0] -> Option D

4. Quick Check:

   4 squared = 16 [OK]

Hint: Polynomial degree 2 fits squares; predict 4^2 = 16 [OK]

Common Mistakes:

• Ignoring polynomial transformation
• Predicting linear value instead of squared
• Rounding errors without np.round

4.

Identify the error in this polynomial regression pipeline code:

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression

pipeline = Pipeline([
    ('linear', LinearRegression()),
    ('poly', PolynomialFeatures(degree=3))
])

pipeline.fit(X_train, y_train)

medium

A. LinearRegression should not be used in pipeline

B. The order of pipeline steps is incorrect

C. PolynomialFeatures degree must be 2, not 3

D. Missing import for X_train and y_train

Solution

1. Step 1: Check pipeline step order

   PolynomialFeatures must come before LinearRegression to transform data first.

2. Step 2: Confirm degree and imports

   Degree 3 is valid; imports for data are assumed outside snippet.

3. Final Answer:

   The order of pipeline steps is incorrect -> Option B

4. Quick Check:

   PolynomialFeatures before LinearRegression [OK]

Hint: PolynomialFeatures must be first in pipeline [OK]

Common Mistakes:

• Swapping order of steps
• Thinking degree must be 2
• Confusing missing data imports with pipeline error

5.

You want to model a dataset with a complex curve. You try polynomial regression with degree=2 but the fit is poor. What is the best next step?

hard

A. Remove polynomial features and use linear regression only

B. Decrease the polynomial degree to avoid overfitting

C. Increase the polynomial degree to capture more complexity

D. Use degree=2 but reduce training data size

Solution

1. Step 1: Understand model complexity and fit

   Degree 2 polynomial may be too simple for complex curves, causing poor fit.

2. Step 2: Adjust polynomial degree

   Increasing degree allows model to fit more complex patterns, improving fit quality.

3. Final Answer:

   Increase the polynomial degree to capture more complexity -> Option C

4. Quick Check:

   Higher degree = better complex fit [OK]

Hint: Raise degree to fit complex curves better [OK]

Common Mistakes:

• Lowering degree when fit is poor
• Removing polynomial features unnecessarily
• Reducing data size instead of model complexity