Polynomial features in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Polynomial Feature Expansion

Which of the following best describes what polynomial features do to the original input data in machine learning?

A They normalize the features to have zero mean and unit variance.

B They create new features by multiplying existing features up to a specified degree, allowing models to learn nonlinear relationships.

C They reduce the number of features by selecting only the most important ones.

D They convert categorical features into numerical codes.

💡 Hint

Think about how polynomial features help linear models capture curves.

Polynomial features generate new features by taking powers and products of the original features, enabling linear models to fit nonlinear patterns.

❓ Predict Output

intermediate

Output of Polynomial Feature Transformation

What is the output of the following Python code using sklearn's PolynomialFeatures with degree=2 on input [[2, 3]]?

ML Python

from sklearn.preprocessing import PolynomialFeatures
import numpy as np
poly = PolynomialFeatures(degree=2, include_bias=False)
X = np.array([[2, 3]])
X_poly = poly.fit_transform(X)
print(X_poly)

A [[2. 3. 5. 6. 9.]]

B [[1. 2. 3. 4. 6. 9.]]

C [[2. 3. 4. 6. 9.]]

D [[2. 3. 4. 5. 6.]]

💡 Hint

Remember include_bias=False means no constant 1 column.

PolynomialFeatures with degree=2 and include_bias=False creates features: original features, their squares, and their product. For input [2,3], these are [2,3,2^2=4,2*3=6,3^2=9].

❓ Model Choice

advanced

Choosing a Model for Polynomial Features

You have expanded your dataset with polynomial features of degree 3. Which model below is most suitable to avoid overfitting and handle the increased feature space?

A Ridge Regression (linear regression with L2 regularization)

B Linear Regression without regularization

C Decision Tree with no depth limit

D K-Nearest Neighbors with k=1

💡 Hint

Think about how to control complexity when features increase.

Ridge Regression adds L2 regularization which helps control overfitting caused by many polynomial features by shrinking coefficients.

❓ Hyperparameter

advanced

Effect of Degree Parameter in Polynomial Features

What is the main effect of increasing the degree parameter in PolynomialFeatures on the dataset?

A It increases the number of features exponentially, which can lead to overfitting.

B It decreases the number of features, simplifying the model.

C It normalizes the features to a smaller range.

D It converts continuous features into categorical bins.

💡 Hint

Think about how many new features are created as degree grows.

Increasing degree creates many new polynomial combinations, increasing feature count exponentially, which can cause overfitting if not controlled.

❓ Metrics

expert

Evaluating Model Performance with Polynomial Features

You trained a polynomial regression model with degree 4 and got training R² = 0.95 but test R² = 0.60. What does this indicate and which metric would best help diagnose the problem?

A The model underfits; use Mean Absolute Error (MAE) to check errors.

B The model is underfitting; use confusion matrix to analyze.

C The model is perfect; use accuracy score to confirm.

D The model overfits; use cross-validation score to assess generalization.

💡 Hint

High train score but low test score usually means overfitting.

High training R² and low test R² means the model fits training data too closely but fails to generalize. Cross-validation helps estimate true performance.

Practice

1. What is the main purpose of using PolynomialFeatures in machine learning?

easy

A. To create new features by adding powers and combinations of existing features

B. To reduce the number of features in the dataset

C. To normalize the data between 0 and 1

D. To split the dataset into training and testing sets

Solution

1. Step 1: Understand the role of PolynomialFeatures

   PolynomialFeatures generates new features by raising existing features to powers and combining them, helping models learn curves.

2. Step 2: Compare with other options

   Feature reduction, normalization between 0 and 1, and splitting into training/testing sets describe different preprocessing steps, not feature creation with powers.

3. Final Answer:

   To create new features by adding powers and combinations of existing features -> Option A

4. Quick Check:

   PolynomialFeatures = create new polynomial features [OK]

Hint: PolynomialFeatures adds powers and combos of features [OK]

Common Mistakes:

• Confusing feature creation with normalization
• Thinking it reduces features instead of expanding
• Mixing it up with data splitting

2. Which of the following is the correct way to import and create polynomial features of degree 2 using scikit-learn?

easy

A. from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2)

B. from sklearn.linear_model import PolynomialFeatures
poly = PolynomialFeatures(2)

C. import PolynomialFeatures from sklearn.preprocessing
poly = PolynomialFeatures(degree=2)

D. from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(3)

Solution

1. Step 1: Check the correct import statement

   PolynomialFeatures is in sklearn.preprocessing, so 'from sklearn.preprocessing import PolynomialFeatures' is correct.

2. Step 2: Verify the degree parameter

   To create degree 2 features, use degree=2 in the constructor.

3. Final Answer:

   from sklearn.preprocessing import PolynomialFeatures poly = PolynomialFeatures(degree=2) -> Option A

4. Quick Check:

   Import from preprocessing and set degree=2 [OK]

Hint: Import from preprocessing and set degree=2 [OK]

Common Mistakes:

• Importing from wrong module
• Forgetting 'degree=' keyword
• Setting wrong degree value

3. Given the code below, what is the output of X_poly?

from sklearn.preprocessing import PolynomialFeatures
import numpy as np
X = np.array([[2, 3]])
poly = PolynomialFeatures(degree=2, include_bias=False)
X_poly = poly.fit_transform(X)
print(X_poly)

medium

A. [[2 3 5 6 9]]

B. [[1 2 3 4 6 9]]

C. [[2 3 4 6 9]]

D. [[2 3 4 5 6 9]]

Solution

1. Step 1: Understand PolynomialFeatures output with degree=2 and include_bias=False

   Features include original features, their squares, and pairwise products: [x1, x2, x1^2, x1*x2, x2^2].

2. Step 2: Calculate values for X = [2, 3]

   x1=2, x2=3; x1^2=4, x1*x2=6, x2^2=9; so output is [[2, 3, 4, 6, 9]].

3. Final Answer:

   [[2 3 4 6 9]] -> Option C

4. Quick Check:

   Polynomial features = original + squares + products [OK]

Hint: Output includes original, squares, and cross-products [OK]

Common Mistakes:

• Including bias term when include_bias=False
• Miscomputing squares or products
• Adding extra features not in degree 2

4. Identify the error in the following code snippet that uses PolynomialFeatures:

from sklearn.preprocessing import PolynomialFeatures
X = [[1, 2], [3, 4]]
poly = PolynomialFeatures(degree=3)
X_poly = poly.fit_transform(X)
print(X_poly)

medium

A. X should be a NumPy array, not a list of lists

B. No error; code runs correctly

C. Missing import for NumPy

D. Degree 3 is not supported by PolynomialFeatures

Solution

1. Step 1: Check input type compatibility

   PolynomialFeatures accepts lists or arrays, so X as list of lists is valid.

2. Step 2: Verify degree parameter and imports

   Degree 3 is supported; no NumPy import needed if not used explicitly.

3. Final Answer:

   No error; code runs correctly -> Option B

4. Quick Check:

   PolynomialFeatures accepts lists and degree 3 [OK]

Hint: PolynomialFeatures accepts lists; degree 3 is valid [OK]

Common Mistakes:

• Assuming input must be NumPy array
• Thinking degree 3 is invalid
• Expecting import errors without NumPy usage

5. You have a dataset with 3 features and want to add polynomial features up to degree 3. How many features will the transformed dataset have if include_bias=False?

hard

A. 10

B. 20

C. 16

D. 19

Solution

1. Step 1: Use formula for number of polynomial features

   Number of features = C(n + d, d) - 1 if include_bias=False, where n=3, d=3.

2. Step 2: Calculate combinations

   C(3+3, 3) = C(6, 3) = 20; subtract 1 for no bias gives 19 features.

3. Final Answer:

   19 -> Option D

4. Quick Check:

   Features = combinations(6,3)-1 = 19 [OK]

Hint: Use combinations(n+d, d) minus bias if excluded [OK]

Common Mistakes:

• Forgetting to subtract bias feature
• Using wrong combination formula
• Confusing degree with number of features