Gradient Boosting (GBM) in ML Python - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to import the GradientBoostingClassifier from scikit-learn.

ML Python

from sklearn.ensemble import [1]

Drag options to blanks, or click blank then click option'

ALogisticRegression

BRandomForestClassifier

CKNeighborsClassifier

DGradientBoostingClassifier

Attempts:

3 left

💡 Hint

Common Mistakes

Importing a different classifier like RandomForestClassifier by mistake.

Confusing GradientBoostingClassifier with LogisticRegression.

The GradientBoostingClassifier is the correct class to import for gradient boosting classification tasks.

2fill in blank

medium

Complete the code to create a GradientBoostingClassifier with 100 trees.

ML Python

model = GradientBoostingClassifier(n_estimators=[1])

Drag options to blanks, or click blank then click option'

A10

B100

C1000

D1

Attempts:

3 left

💡 Hint

Common Mistakes

Using too few trees like 1 or 10 which may underfit.

Using too many trees like 1000 which may be slow to train.

The n_estimators parameter sets the number of boosting stages or trees. 100 is a common default value.

3fill in blank

hard

Fix the error in the code to fit the model on training data X_train and y_train.

ML Python

model.[1](X_train, y_train)

Drag options to blanks, or click blank then click option'

Afit

Bpredict

Ctransform

Dscore

Attempts:

3 left

💡 Hint

Common Mistakes

Using predict instead of fit to train the model.

Using transform which is for data preprocessing.

The fit method trains the model on the given data.

4fill in blank

hard

Fill both blanks to create a dictionary of feature importances from the trained model.

ML Python

feature_importances = {feature: model.[1][index] for index, feature in enumerate([2])}

Drag options to blanks, or click blank then click option'

Afeature_importances_

Bfeature_names

Cfeatures

Dcoef_

Attempts:

3 left

💡 Hint

Common Mistakes

Using coef_ which is for linear models, not gradient boosting.

Using features which is not a standard variable name.

The trained model stores feature importances in feature_importances_. The list of feature names is typically stored in feature_names.

5fill in blank

hard

Fill all three blanks to compute accuracy score of the model on test data.

ML Python

from sklearn.metrics import [1]
predictions = model.[2](X_test)
accuracy = [3](y_test, predictions)

Drag options to blanks, or click blank then click option'

Aaccuracy_score

Bpredict

Dconfusion_matrix

Attempts:

3 left

💡 Hint

Common Mistakes

Using confusion_matrix instead of accuracy_score for metric.

Calling score method instead of using accuracy_score function.

We import accuracy_score to measure accuracy. We use predict to get model predictions. Then we compute accuracy by comparing true labels and predictions.

Practice

1. What is the main idea behind Gradient Boosting (GBM)?

easy

A. Using a single deep neural network for prediction

B. Combining many weak models to create a strong model

C. Clustering data points into groups

D. Reducing data dimensions using PCA

Solution

1. Step 1: Understand the concept of boosting

   Boosting means combining many simple models (weak learners) to improve overall prediction.

2. Step 2: Identify Gradient Boosting's approach

   Gradient Boosting builds models sequentially, each correcting errors of the previous one, making a strong model.

3. Final Answer:

   Combining many weak models to create a strong model -> Option B

4. Quick Check:

   Boosting = Combining weak models [OK]

Hint: Boosting means many weak models combined [OK]

Common Mistakes:

• Confusing boosting with deep learning
• Thinking GBM clusters data
• Mixing boosting with dimensionality reduction

2. Which of the following is the correct way to import GradientBoostingClassifier from scikit-learn?

easy

A. import GradientBoostingClassifier from sklearn

B. from sklearn import GradientBoostingClassifier

C. from sklearn.ensemble import GradientBoostingClassifier

D. import GradientBoostingClassifier from sklearn.ensemble

Solution

1. Step 1: Recall correct import syntax in Python

   Python imports classes or functions using 'from module import class' syntax.

2. Step 2: Identify the correct module for GradientBoostingClassifier

   GradientBoostingClassifier is in sklearn.ensemble, so correct import is from sklearn.ensemble import GradientBoostingClassifier.

3. Final Answer:

   from sklearn.ensemble import GradientBoostingClassifier -> Option C

4. Quick Check:

   Correct import syntax = from sklearn.ensemble import GradientBoostingClassifier [OK]

Hint: Use 'from sklearn.ensemble import GradientBoostingClassifier' [OK]

Common Mistakes:

• Using 'import' instead of 'from ... import ...'
• Importing from wrong module
• Wrong order of import statement

3. What will be the output of the following code snippet?

from sklearn.ensemble import GradientBoostingRegressor
X = [[1], [2], [3], [4]]
y = [2, 4, 6, 8]
gbm = GradientBoostingRegressor(n_estimators=100, learning_rate=0.1)
gbm.fit(X, y)
pred = gbm.predict([[5]])
print(round(pred[0], 1))

medium

A. 9.0

B. 10.0

C. 8.0

D. 6.0

Solution

1. Step 1: Understand the training data and model

   X and y show a linear relation y = 2 * x. The model is GradientBoostingRegressor with 100 trees and learning rate 0.1.

2. Step 2: Predict for input 5

   Gradient Boosting can extrapolate somewhat beyond training data, especially with many estimators and moderate learning rate, so prediction is close to 10.0.

3. Final Answer:

   9.0 -> Option A

4. Quick Check:

   Prediction near linear extrapolation = 9.0 [OK]

Hint: Tree boosting can approximate linear extrapolation with enough estimators [OK]

Common Mistakes:

• Expecting exact linear output
• Ignoring learning rate effect
• Confusing classification with regression output

4. Identify the error in this Gradient Boosting code snippet:

from sklearn.ensemble import GradientBoostingClassifier
X = [[0], [1], [2]]
y = [0, 1, 0]
gbm = GradientBoostingClassifier(n_estimators='100')
gbm.fit(X, y)

medium

A. n_estimators should be an integer, not a string

B. X should be a numpy array, not a list

C. GradientBoostingClassifier cannot handle binary targets

D. Missing learning_rate parameter

Solution

1. Step 1: Check parameter types

   n_estimators expects an integer number of trees, but '100' is a string, causing a type error.

2. Step 2: Validate other parts

   X as list is acceptable, binary targets are valid, learning_rate is optional with default 0.1.

3. Final Answer:

   n_estimators should be an integer, not a string -> Option A

4. Quick Check:

   Parameter types must match expected types [OK]

Hint: Check parameter types carefully [OK]

Common Mistakes:

• Passing numbers as strings
• Assuming lists are invalid input
• Thinking learning_rate is mandatory

5. You want to improve a Gradient Boosting model's accuracy but training is very slow. Which combination of hyperparameters is best to try first?

hard

A. Increase n_estimators and decrease learning_rate

B. Increase both n_estimators and learning_rate

C. Set n_estimators to 1 and learning_rate to 0.01

D. Decrease n_estimators and increase learning_rate

Solution

1. Step 1: Understand hyperparameter effects

   More n_estimators means more trees and slower training; higher learning_rate speeds learning but risks overfitting.

2. Step 2: Balance speed and accuracy

   Decreasing n_estimators reduces training time; increasing learning_rate compensates to keep accuracy.

3. Final Answer:

   Decrease n_estimators and increase learning_rate -> Option D

4. Quick Check:

   Fewer trees + higher learning rate = faster training [OK]

Hint: Fewer trees + higher learning rate speeds training [OK]

Common Mistakes:

• Increasing both slows training
• Too low n_estimators hurts accuracy
• Too low learning_rate slows learning