Gradient Boosting for regression in ML Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the main idea behind Gradient Boosting for regression?

Gradient Boosting builds a strong prediction model by combining many weak models, usually decision trees, where each new model corrects the errors of the previous ones.

beginner

How does Gradient Boosting improve the model step-by-step?

It fits a new model to the residual errors (differences between actual and predicted values) of the previous model, gradually reducing the overall error.

intermediate

What role does the learning rate play in Gradient Boosting for regression?

The learning rate controls how much each new model influences the overall prediction. A smaller learning rate means slower learning but can lead to better accuracy.

beginner

Why are decision trees commonly used as weak learners in Gradient Boosting?

Decision trees are simple, fast to train, and can capture non-linear relationships, making them effective weak learners to be combined in Gradient Boosting.

beginner

What metric is commonly used to evaluate Gradient Boosting regression models?

Common metrics include Mean Squared Error (MSE) and Root Mean Squared Error (RMSE), which measure how close the predicted values are to the actual values.

In Gradient Boosting for regression, what does each new model try to predict?

A The residual errors of the previous model

B The original target values directly

C Random noise in the data

D The average of all previous predictions

Each new model is trained to predict the residual errors (differences between actual and predicted values) from the previous model.

What happens if the learning rate in Gradient Boosting is set too high?

A The model will ignore residuals

B The model will learn too slowly

C The model may overfit and learn too quickly

D The model will stop training

A high learning rate can cause the model to overfit by making large updates that fit noise instead of true patterns.

Which of the following is NOT a typical characteristic of weak learners in Gradient Boosting?

A Strong individual predictive power

B Simple and shallow decision trees

C Fast to train

D Able to capture some patterns

Weak learners have limited predictive power individually; their strength comes from being combined.

Which metric would you use to measure the accuracy of a Gradient Boosting regression model?

A Confusion matrix

B Accuracy score

C Precision

D Mean Squared Error (MSE)

MSE is a common metric for regression tasks, measuring the average squared difference between predicted and actual values.

What is the main benefit of combining many weak models in Gradient Boosting?

A To avoid using decision trees

B To create a strong model with better predictions

C To make the model run faster

D To reduce the size of the dataset

Combining weak models helps reduce errors and improves overall prediction accuracy.

Explain how Gradient Boosting builds a regression model step-by-step.

Think about how each new model fixes mistakes from before.

Describe the role of learning rate and weak learners in Gradient Boosting for regression.

Consider how small steps and simple models work together.