Polynomial regression in ML Python

Introduction

Polynomial regression helps us find curved lines that fit data better than straight lines. It models relationships that are not just straight but can bend.

When data points form a curve instead of a straight line.

To predict values that change in a non-linear way, like growth rates or temperature changes.

When simple linear regression does not fit the data well.

To capture patterns like U-shapes or hills in data.

When you want a simple model that can still handle curves.

Syntax

ML Python

from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import make_pipeline

degree = 2
model = make_pipeline(PolynomialFeatures(degree), LinearRegression())
model.fit(X_train, y_train)
predictions = model.predict(X_test)

degree controls how curved the model is. Degree 1 means a straight line, degree 2 means a curve like a U or hill.

Using make_pipeline helps combine polynomial feature creation and linear regression in one step.

Examples

This creates a model that fits a quadratic curve (degree 2) to the data.

ML Python

model = make_pipeline(PolynomialFeatures(2), LinearRegression())

This fits a cubic curve (degree 3), which can bend more than degree 2.

ML Python

model = make_pipeline(PolynomialFeatures(3), LinearRegression())

Train the polynomial regression model and get predictions on new data.

ML Python

model.fit(X_train, y_train)
predictions = model.predict(X_test)

Sample Program

This example fits a polynomial regression model to data that follows y = x². It trains on the first 7 points and tests on the last 3. It prints predictions and how well the model fits.

ML Python

import numpy as np
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import make_pipeline
from sklearn.metrics import mean_squared_error, r2_score

# Sample data: X is input, y is output with a curve
X = np.array([[1], [2], [3], [4], [5], [6], [7], [8], [9], [10]])
y = np.array([1, 4, 9, 16, 25, 36, 49, 64, 81, 100])  # y = x^2

# Split data into training and testing
X_train, X_test = X[:7], X[7:]
y_train, y_test = y[:7], y[7:]

# Create polynomial regression model with degree 2
model = make_pipeline(PolynomialFeatures(2), LinearRegression())

# Train the model
model.fit(X_train, y_train)

# Predict on test data
predictions = model.predict(X_test)

# Calculate metrics
mse = mean_squared_error(y_test, predictions)
r2 = r2_score(y_test, predictions)

# Print results
print("Predictions:", predictions)
print(f"Mean Squared Error: {mse:.2f}")
print(f"R2 Score: {r2:.2f}")

Important Notes

Choosing a very high degree can cause the model to fit noise, not just the pattern. This is called overfitting.

Polynomial regression is still a linear model but on transformed features (powers of inputs).

Always check model performance on test data to avoid overfitting.

Summary

Polynomial regression fits curved lines to data by adding powers of input features.

It is useful when data shows non-linear patterns like curves or hills.

Use PolynomialFeatures with LinearRegression to create the model easily.