XGBoost in ML Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does XGBoost stand for and what is its main purpose?

XGBoost stands for eXtreme Gradient Boosting. It is a machine learning method used to build strong predictive models by combining many weak models, usually decision trees, to improve accuracy.

beginner

How does XGBoost improve model performance compared to a single decision tree?

XGBoost builds many trees sequentially. Each new tree tries to fix errors made by previous trees. This process, called boosting, helps the model learn from mistakes and become more accurate.

intermediate

What is the role of the learning rate in XGBoost?

The learning rate controls how much each new tree influences the overall model. A smaller learning rate means the model learns slowly but can be more accurate, while a larger rate learns faster but risks overfitting.

intermediate

Explain the concept of regularization in XGBoost.

Regularization in XGBoost adds a penalty for complex trees to prevent overfitting. It helps the model generalize better to new data by keeping trees simpler and avoiding fitting noise.

beginner

What metrics can you use to evaluate an XGBoost model for classification?

Common metrics include accuracy (how many predictions are correct), precision (correct positive predictions), recall (how many actual positives were found), and AUC-ROC (how well the model separates classes).

What type of models does XGBoost primarily use to build its ensemble?

A Decision trees

B Neural networks

C Support vector machines

D K-nearest neighbors

XGBoost builds an ensemble of decision trees sequentially to improve predictions.

What is the main goal of boosting in XGBoost?

A To speed up training by using fewer trees

B To reduce the size of the dataset

C To combine weak models to create a strong model

D To cluster data points

Boosting combines many weak models, like small trees, to make a stronger, more accurate model.

Which hyperparameter in XGBoost controls how much each tree contributes to the final prediction?

A subsample

B max_depth

C n_estimators

D learning_rate

The learning_rate controls the contribution of each new tree to the overall model.

What does regularization help prevent in XGBoost models?

A Underfitting

B Overfitting

C Data leakage

D Data imbalance

Regularization adds penalties to complex models to avoid overfitting the training data.

Which metric is best to evaluate how well an XGBoost model separates two classes?

A AUC-ROC

B Mean squared error

C Accuracy

D Confusion matrix

AUC-ROC measures how well the model distinguishes between classes across different thresholds.

Describe how XGBoost builds its model step-by-step and why this helps improve accuracy.

Think about how each new tree learns from mistakes of earlier trees.

Explain the importance of tuning hyperparameters like learning rate and max_depth in XGBoost.

Consider how these settings affect learning speed and model complexity.