Polynomial regression pipeline in ML Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is polynomial regression?

Polynomial regression is a type of regression analysis where the relationship between the input variable and the output variable is modeled as an nth degree polynomial. It helps capture curved patterns in data.

beginner

Why do we use a pipeline in polynomial regression?

A pipeline helps combine multiple steps like transforming features into polynomial features and fitting a regression model into one sequence. This makes the process cleaner, easier to manage, and reduces errors.

intermediate

What does the PolynomialFeatures transformer do in a pipeline?

PolynomialFeatures creates new features by raising the original features to different powers up to the specified degree. This allows the model to learn nonlinear relationships.

beginner

How do you evaluate the performance of a polynomial regression model?

You can evaluate it using metrics like Mean Squared Error (MSE) or R-squared (R²). MSE measures average squared errors, while R² shows how well the model explains the data variance.

intermediate

What is the risk of using a very high degree polynomial in regression?

Using a very high degree polynomial can cause overfitting, where the model fits the training data too closely and performs poorly on new data. It captures noise instead of the true pattern.

What is the main purpose of PolynomialFeatures in a regression pipeline?

A To normalize the data

B To create new features by raising inputs to powers

C To reduce the number of features

D To split data into training and testing sets

PolynomialFeatures generates new features by raising the original features to different powers, enabling the model to learn nonlinear relationships.

Which metric is commonly used to measure the error of a polynomial regression model?

A Mean Squared Error (MSE)

B Accuracy

C Precision

D Recall

Mean Squared Error (MSE) measures the average squared difference between predicted and actual values, making it suitable for regression error measurement.

What happens if you choose a polynomial degree that is too high?

A The model becomes a linear regression

B The model will always perform better

C The model ignores nonlinear patterns

D The model may overfit the training data

A very high polynomial degree can cause overfitting, where the model fits noise in the training data and performs poorly on new data.

Why is using a pipeline helpful in polynomial regression?

A It visualizes the data

B It automatically tunes hyperparameters

C It combines feature transformation and model fitting steps

D It splits data into batches

A pipeline combines steps like polynomial feature creation and regression fitting into one process, making the workflow simpler and less error-prone.

Which of these is NOT a step in a polynomial regression pipeline?

A Data encryption

B Linear regression fitting

C Polynomial feature transformation

D Model evaluation

Data encryption is unrelated to polynomial regression pipelines, which focus on feature transformation, model fitting, and evaluation.

Explain how a polynomial regression pipeline works from raw data to predictions.

Think about the steps you take to prepare data, train the model, and check results.

Describe the risks and benefits of increasing the polynomial degree in regression.

Consider what happens when the model becomes too simple or too complex.

Practice

1.

What is the main purpose of using polynomial regression instead of simple linear regression?

easy

A. To fit curved relationships between variables

B. To reduce the number of features

C. To speed up training time

D. To handle missing data automatically

Solution

1. Step 1: Understand linear regression limitation

   Linear regression fits straight lines, which cannot capture curves in data.

2. Step 2: Role of polynomial regression

   Polynomial regression fits curved lines by adding powers of features, capturing non-linear patterns.

3. Final Answer:

   To fit curved relationships between variables -> Option A

4. Quick Check:

   Polynomial regression = curved fit [OK]

Hint: Polynomial regression fits curves, not just straight lines [OK]

Common Mistakes:

• Thinking polynomial regression reduces features
• Assuming it speeds up training
• Believing it handles missing data automatically

2.

Which of the following is the correct way to create a polynomial regression pipeline in Python using sklearn?

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression

pipeline = Pipeline([
    ('poly', PolynomialFeatures(degree=2)),
    ('linear', LinearRegression())
])

easy

A. pipeline = Pipeline([('poly', PolynomialFeatures(degree=2)), ('linear', LinearRegression())])

B. pipeline = Pipeline([('linear', LinearRegression()), ('poly', PolynomialFeatures(degree=2))])

C. pipeline = Pipeline([('poly', LinearRegression()), ('linear', PolynomialFeatures(degree=2))])

D. pipeline = Pipeline([('poly', PolynomialFeatures()), ('linear', LinearRegression(degree=2))])

Solution

1. Step 1: Order of pipeline steps

   PolynomialFeatures must come before LinearRegression to transform data first.

2. Step 2: Correct usage of classes and parameters

   PolynomialFeatures takes degree parameter; LinearRegression does not take degree.

3. Final Answer:

   pipeline = Pipeline([('poly', PolynomialFeatures(degree=2)), ('linear', LinearRegression())]) -> Option A

4. Quick Check:

   PolynomialFeatures before LinearRegression [OK]

Hint: Put PolynomialFeatures before LinearRegression in pipeline [OK]

Common Mistakes:

• Swapping order of pipeline steps
• Passing degree to LinearRegression
• Omitting degree in PolynomialFeatures

3.

Given the following code, what will print(y_pred) output?

import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression

X = np.array([[1], [2], [3]])
y = np.array([1, 4, 9])

pipeline = Pipeline([
    ('poly', PolynomialFeatures(degree=2)),
    ('linear', LinearRegression())
])
pipeline.fit(X, y)
y_pred = pipeline.predict(np.array([[4]]))
print(np.round(y_pred, 2))

medium

A. [10.0]

B. [8.0]

C. [4.0]

D. [16.0]

Solution

1. Step 1: Understand data and model

   X = [[1],[2],[3]] with y = [1,4,9] fits y = x^2 perfectly.

2. Step 2: Predict for X=4 using polynomial degree 2

   Model learns y = x^2, so prediction at 4 is 4^2 = 16.

3. Final Answer:

   [16.0] -> Option D

4. Quick Check:

   4 squared = 16 [OK]

Hint: Polynomial degree 2 fits squares; predict 4^2 = 16 [OK]

Common Mistakes:

• Ignoring polynomial transformation
• Predicting linear value instead of squared
• Rounding errors without np.round

4.

Identify the error in this polynomial regression pipeline code:

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression

pipeline = Pipeline([
    ('linear', LinearRegression()),
    ('poly', PolynomialFeatures(degree=3))
])

pipeline.fit(X_train, y_train)

medium

A. LinearRegression should not be used in pipeline

B. The order of pipeline steps is incorrect

C. PolynomialFeatures degree must be 2, not 3

D. Missing import for X_train and y_train

Solution

1. Step 1: Check pipeline step order

   PolynomialFeatures must come before LinearRegression to transform data first.

2. Step 2: Confirm degree and imports

   Degree 3 is valid; imports for data are assumed outside snippet.

3. Final Answer:

   The order of pipeline steps is incorrect -> Option B

4. Quick Check:

   PolynomialFeatures before LinearRegression [OK]

Hint: PolynomialFeatures must be first in pipeline [OK]

Common Mistakes:

• Swapping order of steps
• Thinking degree must be 2
• Confusing missing data imports with pipeline error

5.

You want to model a dataset with a complex curve. You try polynomial regression with degree=2 but the fit is poor. What is the best next step?

hard

A. Remove polynomial features and use linear regression only

B. Decrease the polynomial degree to avoid overfitting

C. Increase the polynomial degree to capture more complexity

D. Use degree=2 but reduce training data size

Solution

1. Step 1: Understand model complexity and fit

   Degree 2 polynomial may be too simple for complex curves, causing poor fit.

2. Step 2: Adjust polynomial degree

   Increasing degree allows model to fit more complex patterns, improving fit quality.

3. Final Answer:

   Increase the polynomial degree to capture more complexity -> Option C

4. Quick Check:

   Higher degree = better complex fit [OK]

Hint: Raise degree to fit complex curves better [OK]

Common Mistakes:

• Lowering degree when fit is poor
• Removing polynomial features unnecessarily
• Reducing data size instead of model complexity