Polynomial features in ML Python

Introduction

Polynomial features help us find patterns that are not straight lines by adding powers of the original data. This lets simple models learn curves and more complex shapes.

When you want to predict something that changes in a curve, like the growth of plants over time.

When your data points don't line up in a straight line but seem to follow a curve.

When you want to improve a simple model by giving it more information about combinations of features.

When you want to capture interactions between features without manually creating new columns.

Syntax

ML Python

from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2, include_bias=False)
X_poly = poly.fit_transform(X)

degree controls the highest power of features to include.

include_bias=False means we don't add a column of ones (intercept).

Examples

Adds original features, their squares, and pairwise products.

ML Python

poly = PolynomialFeatures(degree=2)
X_poly = poly.fit_transform(X)

Adds features up to cubes without the bias column.

ML Python

poly = PolynomialFeatures(degree=3, include_bias=False)
X_poly = poly.fit_transform(X)

Returns the original features plus a bias column (if include_bias=True).

ML Python

poly = PolynomialFeatures(degree=1)
X_poly = poly.fit_transform(X)

Sample Model

This example shows how to turn two features into polynomial features, train a simple model, and check how well it fits.

ML Python

from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
import numpy as np

# Sample data: hours studied and hours slept
X = np.array([[1, 2], [2, 3], [3, 4], [4, 5]])
y = np.array([5, 7, 9, 11])  # target: test score

# Create polynomial features of degree 2
poly = PolynomialFeatures(degree=2, include_bias=False)
X_poly = poly.fit_transform(X)

# Train linear regression on polynomial features
model = LinearRegression()
model.fit(X_poly, y)

# Predict on training data
y_pred = model.predict(X_poly)

# Calculate mean squared error
mse = mean_squared_error(y, y_pred)

print(f"Original features:\n{X}")
print(f"Polynomial features:\n{X_poly}")
print(f"Predictions: {y_pred}")
print(f"Mean Squared Error: {mse:.4f}")

Important Notes

Polynomial features can increase the number of columns quickly, so use small degrees for many features.

Always scale your data before polynomial transformation if features have very different scales.

Higher degree polynomials can cause overfitting, so test carefully.

Summary

Polynomial features add powers and combinations of original features to help models learn curves.

Use PolynomialFeatures from scikit-learn to create these new features easily.

Be careful with degree size to avoid too many features or overfitting.