Polynomial features in Data Analysis Python - Step-by-Step Execution

Concept Flow - Polynomial features

Start with original features

↓

Choose polynomial degree

↓

Generate new features by multiplying original features

↓

Add interaction terms (products of different features)

↓

Combine all polynomial features into new dataset

↓

Use new features for modeling

We start with original data features, pick a polynomial degree, create new features by multiplying originals and their combinations, then use these new features for analysis.

Execution Sample

Data Analysis Python

from sklearn.preprocessing import PolynomialFeatures
import pandas as pd

X = pd.DataFrame({'x1': [1, 2], 'x2': [3, 4]})
poly = PolynomialFeatures(degree=2, include_bias=False)
X_poly = poly.fit_transform(X)

This code creates polynomial features of degree 2 from two original features x1 and x2.

Execution Table

Step	Input Features	Degree	Generated Features	Output Shape
1	{'x1': 1, 'x2': 3}	2	[x1=1, x2=3, x1^2=1, x1*x2=3, x2^2=9]	(1, 5)
2	{'x1': 2, 'x2': 4}	2	[x1=2, x2=4, x1^2=4, x1*x2=8, x2^2=16]	(1, 5)
3	All rows processed	2	Combined polynomial features for all samples	(2, 5)
4	End	-	-	Transformation complete

💡 All input rows processed and polynomial features generated for degree 2

Variable Tracker

Variable	Start	After 1	After 2	Final
X	{'x1': [1,2], 'x2': [3,4]}	{'x1': 1, 'x2': 3}	{'x1': 2, 'x2': 4}	{'x1': [1,2], 'x2': [3,4]}
X_poly	None	[1, 3, 1, 3, 9]	[2, 4, 4, 8, 16]	[[1,3,1,3,9],[2,4,4,8,16]]

Key Moments - 3 Insights

Why does the output have more features than the input?

Because polynomial features include original features, their squares, and interaction terms (products), increasing the total number of features as shown in execution_table rows 1 and 2.

What does include_bias=False mean in PolynomialFeatures?

It means the output does not include a constant feature of 1 (bias term). This is why the output features start with original features and their combinations, not a column of ones.

How does degree affect the number of generated features?

Higher degree means more combinations and powers of features, so the number of generated features grows quickly, as seen by the polynomial terms in execution_table rows 1 and 2.

Visual Quiz - 3 Questions

Look at the execution_table at Step 1, what is the value of the interaction term x1*x2?

A 1

B 9

C 3

D 0

💡 Hint

Check the Generated Features column at Step 1 in the execution_table.

At which step does the transformation process complete for all input rows?

A Step 3

B Step 1

C Step 2

D Step 4

💡 Hint

Look for the step mentioning 'All rows processed' in the execution_table.

If we set degree=1, how would the number of generated features change compared to degree=2?

A It would be the same number of features

B It would be fewer features, only original ones

C It would be more features, including squares

D It would include only interaction terms

💡 Hint

Refer to the concept_snapshot about degree and feature generation.

Concept Snapshot

PolynomialFeatures creates new features by raising original features to powers and multiplying them.
Syntax: PolynomialFeatures(degree=n, include_bias=False).
Degree controls max power and interaction terms.
Output shape grows with degree and number of original features.
Useful to capture nonlinear relationships in data.

Full Transcript

Polynomial features start from original data columns. We pick a degree, like 2, to create new features by multiplying and squaring originals. For example, with features x1 and x2, degree 2 creates x1, x2, x1 squared, x1 times x2, and x2 squared. This increases the number of features, helping models learn more complex patterns. The code example uses sklearn's PolynomialFeatures to do this transformation. The execution table shows step-by-step how each row's features expand. Key points include why output features increase, what include_bias means, and how degree affects feature count. The visual quiz checks understanding of interaction terms, process steps, and degree effects. The snapshot summarizes usage and behavior for quick reference.

Practice

1. What is the main purpose of using Polynomial features in data analysis?

easy

A. To reduce the number of features in the dataset

B. To capture non-linear relationships by adding powers and combinations of features

C. To remove outliers from the data

D. To convert categorical data into numerical data

Solution

1. Step 1: Understand what polynomial features do

   Polynomial features create new features by raising existing features to powers and combining them, which helps model curves.

2. Step 2: Identify the purpose in modeling

   This helps capture non-linear relationships that simple linear features cannot represent.

3. Final Answer:

   To capture non-linear relationships by adding powers and combinations of features -> Option B

4. Quick Check:

   Polynomial features = capture curves [OK]

Hint: Polynomial features add powers to model curves, not reduce features [OK]

Common Mistakes:

• Thinking polynomial features reduce feature count
• Confusing polynomial features with data cleaning
• Assuming polynomial features handle categorical data

2. Which of the following is the correct way to create polynomial features of degree 2 using scikit-learn in Python?

easy

A. from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(1)
X_poly = poly.transform(X)

B. from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2)
X_poly = poly.fit(X)

C. from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=3)
X_poly = poly.fit_transform(X)

D. from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2)
X_poly = poly.fit_transform(X)

Solution

1. Step 1: Import and initialize PolynomialFeatures correctly

   Use degree=2 to specify the polynomial degree.

2. Step 2: Use fit_transform to fit and transform the data in one step

   fit_transform returns the transformed feature matrix, which is needed for modeling.

3. Final Answer:

   from sklearn.preprocessing import PolynomialFeatures poly = PolynomialFeatures(degree=2) X_poly = poly.fit_transform(X) -> Option D

4. Quick Check:

   Use degree=2 and fit_transform() [OK]

Hint: Use fit_transform() with degree=2 to create polynomial features [OK]

Common Mistakes:

• Using transform() without fitting first
• Setting wrong degree value
• Calling fit() without transform or fit_transform()

3. Given the code below, what is the shape of X_poly?

import numpy as np
from sklearn.preprocessing import PolynomialFeatures
X = np.array([[2, 3]])
poly = PolynomialFeatures(degree=2)
X_poly = poly.fit_transform(X)

medium

A. (1, 6)

B. (1, 3)

C. (2, 3)

D. (2, 6)

Solution

1. Step 1: Count original features and polynomial features

   Original X has 2 features. Degree 2 polynomial features include: - bias (1) - original features (2) - squared features and interaction (3): x1^2, x1*x2, x2^2 Total = 1 + 2 + 3 = 6 features.

2. Step 2: Check number of samples

   X has 1 sample (row), so output shape is (1, 6).

3. Final Answer:

   (1, 6) -> Option A

4. Quick Check:

   Features after degree 2 = 6 [OK]

Hint: Degree 2 with 2 features gives 6 output features [OK]

Common Mistakes:

• Ignoring bias (constant) term
• Counting only squared terms, missing interactions
• Confusing samples with features in shape

4. Identify the error in the following code that tries to create polynomial features of degree 3:

from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(3)
X_poly = poly.fit_transform([[1, 2], [3, 4]])

medium

A. PolynomialFeatures requires degree to be specified as a keyword argument

B. fit_transform cannot be used on lists, input must be a numpy array

C. No error, the code runs correctly

D. PolynomialFeatures does not support degree 3

Solution

1. Step 1: Check PolynomialFeatures initialization

   Degree can be passed as positional argument, so PolynomialFeatures(3) is valid.

2. Step 2: Check input type for fit_transform

   fit_transform accepts array-like inputs, including lists, so [[1, 2], [3, 4]] is valid.

3. Final Answer:

   No error, the code runs correctly -> Option C

4. Quick Check:

   Degree 3 and list input are valid [OK]

Hint: PolynomialFeatures accepts degree as positional and list input [OK]

Common Mistakes:

• Thinking degree must be keyword argument
• Assuming lists are invalid input
• Believing degree 3 is unsupported

5. You have a dataset with 3 features and want to add polynomial features up to degree 2 but exclude the bias (constant) term. Which code snippet correctly achieves this?

hard

A. poly = PolynomialFeatures(degree=2, include_bias=False)
X_poly = poly.fit_transform(X)

B. poly = PolynomialFeatures(degree=2, include_bias=True)
X_poly = poly.fit_transform(X)
X_poly = X_poly[:, :1]

C. poly = PolynomialFeatures(degree=2)
X_poly = poly.fit_transform(X)
X_poly = X_poly[:, :1]

D. poly = PolynomialFeatures(degree=2, bias=False)
X_poly = poly.fit_transform(X)

Solution

1. Step 1: Understand the include_bias parameter

   Setting include_bias=False excludes the constant (bias) term from the output features.

2. Step 2: Use correct parameter and method

   poly = PolynomialFeatures(degree=2, include_bias=False) and then fit_transform(X) correctly creates polynomial features without bias.

3. Final Answer:

   poly = PolynomialFeatures(degree=2, include_bias=False) X_poly = poly.fit_transform(X) -> Option A

4. Quick Check:

   include_bias=False excludes constant term [OK]

Hint: Use include_bias=False to skip constant term [OK]

Common Mistakes:

• Trying to remove bias by slicing output array
• Using wrong parameter name like bias=False
• Assuming include_bias defaults to False