Gradient Boosting for regression in ML Python - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to import the GradientBoostingRegressor from scikit-learn.

ML Python

from sklearn.ensemble import [1]

Drag options to blanks, or click blank then click option'

ARandomForestRegressor

BGradientBoostingRegressor

CLinearRegression

DKNeighborsRegressor

Attempts:

3 left

💡 Hint

Common Mistakes

Importing a classifier instead of a regressor.

Importing a different ensemble model like RandomForestRegressor.

The GradientBoostingRegressor is the correct class to import for gradient boosting regression models in scikit-learn.

2fill in blank

medium

Complete the code to create a Gradient Boosting regressor with 100 trees.

ML Python

model = GradientBoostingRegressor(n_estimators=[1])

Drag options to blanks, or click blank then click option'

A100

B10

C1000

D1

Attempts:

3 left

💡 Hint

Common Mistakes

Using too few trees like 1 or 10 which may underfit.

Using too many trees like 1000 which may be slow to train.

Setting n_estimators=100 creates a model with 100 boosting stages (trees), which is a common default.

3fill in blank

hard

Fix the error in the code to fit the model on training data X_train and y_train.

ML Python

model.[1](X_train, y_train)

Drag options to blanks, or click blank then click option'

Afit

Bpredict

Ctransform

Dscore

Attempts:

3 left

💡 Hint

Common Mistakes

Calling predict before fitting the model.

Using transform which is for data preprocessing.

The fit method trains the model on the given data. Using predict or transform here is incorrect.

4fill in blank

hard

Fill both blanks to compute the mean squared error between true and predicted values.

ML Python

from sklearn.metrics import [1]
error = [2](y_true, y_pred)

Drag options to blanks, or click blank then click option'

Amean_squared_error

Baccuracy_score

Cmean_absolute_error

Dr2_score

Attempts:

3 left

💡 Hint

Common Mistakes

Using accuracy_score which is for classification.

Using mean_absolute_error which measures absolute errors, not squared.

The mean_squared_error function computes the average squared difference between true and predicted values, which is a common regression metric.

5fill in blank

hard

Fill all three blanks to create a Gradient Boosting regressor with learning rate 0.1, max depth 3, and fit it on data.

ML Python

model = GradientBoostingRegressor(learning_rate=[1], max_depth=[2])
model.[3](X_train, y_train)

Drag options to blanks, or click blank then click option'

A0.1

B3

Cfit

Dpredict

Attempts:

3 left

💡 Hint

Common Mistakes

Using predict instead of fit to train the model.

Setting max_depth too high or learning_rate too large.

Setting learning_rate=0.1 controls the step size at each boosting iteration. max_depth=3 limits tree depth to avoid overfitting. The fit method trains the model.

Practice

1. What is the main idea behind Gradient Boosting for regression?

easy

A. Combining many simple models step-by-step to improve predictions

B. Using a single complex model to predict values

C. Randomly guessing values and selecting the best guess

D. Using only one decision tree without updates

Solution

1. Step 1: Understand Gradient Boosting concept

   Gradient Boosting builds a strong model by adding simple models one after another, each fixing errors of the previous.

2. Step 2: Compare options with this idea

   Only Combining many simple models step-by-step to improve predictions describes combining many simple models step-by-step to improve predictions.

3. Final Answer:

   Combining many simple models step-by-step to improve predictions -> Option A

4. Quick Check:

   Gradient Boosting = Combining simple models [OK]

Hint: Remember: Gradient Boosting adds models one by one [OK]

Common Mistakes:

• Thinking it uses only one model
• Confusing with random guessing
• Assuming it uses a single complex model

2. Which of the following is the correct way to create a Gradient Boosting Regressor in Python using scikit-learn?

easy

A. import GradientBoostingRegressor
model = GradientBoostingRegressor()

B. from sklearn.linear_model import GradientBoostingRegressor
model = GradientBoostingRegressor(learning_rate=0.1)

C. from sklearn.ensemble import GradientBoostingClassifier
model = GradientBoostingClassifier(n_estimators=100)

D. from sklearn.ensemble import GradientBoostingRegressor
model = GradientBoostingRegressor(n_estimators=100, learning_rate=0.1)

Solution

1. Step 1: Identify correct import and class for regression

   GradientBoostingRegressor is in sklearn.ensemble, not sklearn.linear_model or a classifier.

2. Step 2: Check syntax correctness

   from sklearn.ensemble import GradientBoostingRegressor model = GradientBoostingRegressor(n_estimators=100, learning_rate=0.1) correctly imports and creates the model with parameters n_estimators and learning_rate.

3. Final Answer:

   Correct import and model creation with sklearn.ensemble.GradientBoostingRegressor -> Option D

4. Quick Check:

   Correct import and class = from sklearn.ensemble import GradientBoostingRegressor model = GradientBoostingRegressor(n_estimators=100, learning_rate=0.1) [OK]

Hint: Use sklearn.ensemble for GradientBoostingRegressor [OK]

Common Mistakes:

• Importing from wrong module
• Using classifier instead of regressor
• Missing parameters or wrong syntax

3. What will be the output of the following code snippet?

from sklearn.ensemble import GradientBoostingRegressor
import numpy as np

X = np.array([[1], [2], [3], [4], [5]])
y = np.array([1.5, 3.5, 5.5, 7.5, 9.5])
model = GradientBoostingRegressor(n_estimators=10, learning_rate=0.5)
model.fit(X, y)
pred = model.predict(np.array([[6]]))
print(round(pred[0], 1))

medium

A. 10.0

B. 11.5

C. 12.0

D. 9.5

Solution

1. Step 1: Understand training data pattern

   y roughly equals 2*x - 0.5 (1.5, 3.5, 5.5, 7.5, 9.5). So for x=6, expected y ~ 11.5.

2. Step 2: Predict with Gradient Boosting model

   Model with 10 estimators and learning rate 0.5 fits this pattern well, predicting close to 11.5 for input 6.

3. Final Answer:

   11.5 -> Option B

4. Quick Check:

   Prediction for 6 ≈ 11.5 [OK]

Hint: Check pattern in y to guess prediction quickly [OK]

Common Mistakes:

• Ignoring the linear pattern in data
• Confusing classifier with regressor output
• Rounding errors or wrong rounding

4. Identify the error in this Gradient Boosting regression code and fix it:

from sklearn.ensemble import GradientBoostingRegressor
X = [[1], [2], [3]]
y = [2, 4, 6]
model = GradientBoostingRegressor(n_estimators=50)
model.fit(X, y)
print(model.predict([4]))

medium

A. Import GradientBoostingClassifier instead

B. Change n_estimators to 1

C. Change predict input to [[4]] instead of [4]

D. Change y to a numpy array

Solution

1. Step 1: Check input shape for predict method

   Model expects 2D array for predict, but [4] is 1D. It should be [[4]] to match training input shape.

2. Step 2: Fix predict input shape

   Changing predict input to [[4]] fixes the error and allows prediction.

3. Final Answer:

   Change predict input to [[4]] instead of [4] -> Option C

4. Quick Check:

   Predict input shape must match training input [OK]

Hint: Always use 2D array for predict input in scikit-learn [OK]

Common Mistakes:

• Passing 1D array to predict
• Changing unrelated parameters
• Using classifier instead of regressor

5. You want to improve your Gradient Boosting regression model's accuracy on a dataset but notice it overfits. Which combination of parameter changes is best to reduce overfitting?

hard

A. Decrease n_estimators and decrease learning_rate

B. Decrease n_estimators and increase learning_rate

C. Increase n_estimators and decrease learning_rate

D. Increase n_estimators and increase learning_rate

Solution

1. Step 1: Understand overfitting in Gradient Boosting

   Overfitting means model fits training data too closely, losing generalization.

2. Step 2: Adjust parameters to reduce overfitting

   Decreasing n_estimators reduces model complexity; decreasing learning_rate slows learning, both help reduce overfitting.

3. Final Answer:

   Decrease n_estimators and decrease learning_rate -> Option A

4. Quick Check:

   Lower complexity and slower learning reduce overfitting [OK]

Hint: Lower n_estimators and learning_rate to fight overfitting [OK]

Common Mistakes:

• Increasing both parameters causing more overfitting
• Increasing learning_rate alone
• Ignoring parameter effects on overfitting