Polynomial regression in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Polynomial Regression Degree

Which statement best describes the role of the degree in polynomial regression?

A The degree controls the highest power of the input features, allowing the model to fit more complex curves.

B The degree determines the number of input features used in the model.

C The degree sets the number of output variables the model predicts.

D The degree limits the number of training samples used during fitting.

💡 Hint

Think about how polynomial terms like x² or x³ affect the shape of the curve.

The degree in polynomial regression defines the highest power of the input variable included. Higher degrees allow the model to capture more complex, curved relationships.

❓ Predict Output

intermediate

Output of Polynomial Feature Transformation

What is the output of the following code snippet?

ML Python

from sklearn.preprocessing import PolynomialFeatures
import numpy as np
X = np.array([[2]])
poly = PolynomialFeatures(degree=3, include_bias=False)
result = poly.fit_transform(X)
print(result)

A [[2. 4. 8.]]

B [[2. 4.]]

C [[2. 3. 4.]]

D [[1. 2. 4. 8.]]

💡 Hint

Remember that include_bias=False excludes the constant 1 term.

PolynomialFeatures with degree=3 and include_bias=False transforms input [2] into [2, 2²=4, 2³=8].

❓ Hyperparameter

advanced

Choosing Degree to Avoid Overfitting

You have a dataset with noisy data points. Which degree choice for polynomial regression is most likely to avoid overfitting?

A Degree 10, because higher degree fits data perfectly.

B Degree 1, because it fits a simple straight line.

C Degree 5, as a moderate complexity balance.

D Degree 0, which fits a constant value.

💡 Hint

Overfitting happens when the model is too complex for the data.

Degree 5 offers a balance between underfitting (too simple) and overfitting (too complex). Degree 10 is likely too complex, degree 1 too simple, and degree 0 too trivial.

❓ Metrics

advanced

Interpreting Polynomial Regression Metrics

After training a polynomial regression model, you get these metrics on test data: Mean Squared Error (MSE) = 0.02, R² = 0.95. What does this tell you?

A The model has high error and poor fit to the data.

B The model fits the data well with low error and explains 95% of variance.

C The model is underfitting because MSE is too low.

D The model explains only 2% of the variance.

💡 Hint

Recall that lower MSE and higher R² indicate better fit.

Low MSE means predictions are close to actual values. R² of 0.95 means 95% of variance is explained by the model, indicating a good fit.

🔧 Debug

expert

Debugging Polynomial Regression Prediction Error

Consider this code snippet for polynomial regression prediction. What is the main reason the predictions are all zeros?

ML Python

from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures
import numpy as np

X_train = np.array([[1], [2], [3], [4]])
y_train = np.array([1, 4, 9, 16])

poly = PolynomialFeatures(degree=2)
X_poly = poly.fit_transform(X_train)

model = LinearRegression()
model.fit(X_train, y_train)

X_test = np.array([[5], [6]])
X_test_poly = poly.transform(X_test)
predictions = model.predict(X_test_poly)
print(predictions)

A The training labels y_train are incorrect for polynomial regression.

B The degree of PolynomialFeatures is too low to capture the data pattern.

C The test data X_test is not transformed before prediction.

D The model was trained on original features, but predictions use polynomial features, causing mismatch.

💡 Hint

Check if the model was trained on the same feature format as used for prediction.

The model was trained on X_train (original features), but prediction uses X_test_poly (polynomial features). This mismatch causes poor predictions (close to zero). The model should be trained on X_poly.

Practice

1. What is the main purpose of polynomial regression in machine learning?

easy

A. To perform clustering on unlabeled data

B. To reduce the number of features in the dataset

C. To classify data into categories

D. To model non-linear relationships by adding powers of input features

Solution

1. Step 1: Understand the goal of polynomial regression

   Polynomial regression extends linear regression by adding powers of input features to capture curves in data.

2. Step 2: Compare options with this goal

   Only To model non-linear relationships by adding powers of input features describes modeling non-linear relationships by adding powers of features, which matches polynomial regression.

3. Final Answer:

   To model non-linear relationships by adding powers of input features -> Option D

4. Quick Check:

   Polynomial regression = model curves [OK]

Hint: Polynomial regression fits curves by adding powers of inputs [OK]

Common Mistakes:

• Confusing polynomial regression with classification
• Thinking it reduces features instead of expanding them
• Mixing it up with clustering methods

2. Which Python code snippet correctly creates polynomial features of degree 3 using scikit-learn?

easy

A. from sklearn.linear_model import PolynomialFeatures
poly = PolynomialFeatures(3)
X_poly = poly.transform(X)

B. from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=3)
X_poly = poly.fit_transform(X)

C. from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2)
X_poly = poly.fit(X)

D. from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=3)
X_poly = poly.transform(X)

Solution

1. Step 1: Check import and class usage

   PolynomialFeatures is in sklearn.preprocessing, so import must be from there.

2. Step 2: Verify method calls and degree

   Degree=3 is correct. fit_transform(X) creates polynomial features properly. from sklearn.preprocessing import PolynomialFeatures poly = PolynomialFeatures(degree=3) X_poly = poly.fit_transform(X) matches all these.

3. Final Answer:

   from sklearn.preprocessing import PolynomialFeatures poly = PolynomialFeatures(degree=3) X_poly = poly.fit_transform(X) -> Option B

4. Quick Check:

   Correct import + degree + fit_transform = A [OK]

Hint: Use fit_transform with PolynomialFeatures from sklearn.preprocessing [OK]

Common Mistakes:

• Importing PolynomialFeatures from wrong module
• Using transform without fit first
• Setting wrong degree value

3. Given this code, what will be the shape of X_poly if X has shape (5, 1) and degree=2?

from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2)
X_poly = poly.fit_transform(X)

medium

A. (5, 1)

B. (5, 2)

C. (5, 3)

D. (5, 4)

Solution

1. Step 1: Understand PolynomialFeatures output shape

   For degree=2 and one feature, output includes [1, x, x^2], so 3 features total.

2. Step 2: Calculate shape based on input rows and new features

   Input has 5 rows, so output shape is (5, 3).

3. Final Answer:

   (5, 3) -> Option C

4. Quick Check:

   Degree 2 with 1 feature = 3 columns [OK]

Hint: Output columns = combinations with degree + 1 for bias [OK]

Common Mistakes:

• Forgetting the bias (constant 1) column
• Assuming output shape equals input shape
• Confusing degree with number of features

4. Identify the error in this polynomial regression code snippet:

from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
poly = PolynomialFeatures(degree=2)
X_poly = poly.transform(X)
model = LinearRegression()
model.fit(X_poly, y)

medium

A. Using transform before fitting PolynomialFeatures

B. LinearRegression cannot be used with polynomial features

C. Missing import for LinearRegression

D. Degree should be 1 for polynomial regression

Solution

1. Step 1: Check usage of PolynomialFeatures methods

   PolynomialFeatures must be fitted before transform; fit_transform fits and transforms in one step.

2. Step 2: Identify the error in code

   Code calls poly.transform(X) without fitting first, causing error. Correct is poly.fit_transform(X).

3. Final Answer:

   Using transform before fitting PolynomialFeatures -> Option A

4. Quick Check:

   fit_transform needed before transform [OK]

Hint: Always fit PolynomialFeatures before transform or use fit_transform [OK]

Common Mistakes:

• Calling transform without fit first
• Thinking LinearRegression can't handle polynomial features
• Ignoring method order in preprocessing

5. You have data with a clear curve pattern. You fit a polynomial regression model with degree 5 and get very low training error but high error on new data. What is the best next step?

hard

A. Reduce the polynomial degree to avoid overfitting

B. Increase the polynomial degree to 10 for better fit

C. Remove polynomial features and use linear regression only

D. Add more polynomial features of degree 6

Solution

1. Step 1: Understand the problem of low training but high test error

   This indicates overfitting: model fits training data too closely but fails to generalize.

2. Step 2: Choose a solution to reduce overfitting

   Reducing polynomial degree simplifies the model, helping it generalize better on new data.

3. Final Answer:

   Reduce the polynomial degree to avoid overfitting -> Option A

4. Quick Check:

   Overfitting? Lower degree polynomial [OK]

Hint: Lower degree to reduce overfitting when test error is high [OK]

Common Mistakes:

• Increasing degree worsens overfitting
• Removing polynomial features loses curve fit
• Adding higher degree features increases complexity