How to Use XGBoost Classifier in Python: Simple Guide

To use the XGBClassifier in Python, first install the xgboost package, then import XGBClassifier from xgboost. Create an instance, fit it on training data with fit(), and predict with predict().

Syntax

The basic syntax to use XGBClassifier involves importing the class, creating a model instance, training it with fit(), and making predictions with predict().

• Import: Import XGBClassifier from the xgboost library.
• Create model: Initialize with optional parameters like n_estimators (number of trees) and max_depth (tree depth).
• Train: Use fit(X_train, y_train) to train on features and labels.
• Predict: Use predict(X_test) to get class predictions.

python

from xgboost import XGBClassifier

model = XGBClassifier(n_estimators=100, max_depth=3)
model.fit(X_train, y_train)
predictions = model.predict(X_test)

Example

This example shows how to train an XGBClassifier on the Iris dataset, then predict and evaluate accuracy.

python

from xgboost import XGBClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Load data
iris = load_iris()
X, y = iris.data, iris.target

# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Create and train model
model = XGBClassifier(use_label_encoder=False, eval_metric='mlogloss')
model.fit(X_train, y_train)

# Predict
predictions = model.predict(X_test)

# Evaluate
accuracy = accuracy_score(y_test, predictions)
print(f"Accuracy: {accuracy:.2f}")

Output

Accuracy: 1.00

Common Pitfalls

• Missing package: Forgetting to install xgboost causes import errors.
• Label encoding: For multi-class, set use_label_encoder=False and specify eval_metric to avoid warnings.
• Data format: Input features must be numeric arrays; categorical data needs encoding first.
• Overfitting: Using too many trees or too deep trees can overfit; tune n_estimators and max_depth.

python

from xgboost import XGBClassifier

# Wrong: missing eval_metric causes warning
model = XGBClassifier(use_label_encoder=False)

# Right: specify eval_metric
model = XGBClassifier(use_label_encoder=False, eval_metric='logloss')

Quick Reference

Parameter	Description	Default
n_estimators	Number of trees to build	100
max_depth	Maximum depth of each tree	6
learning_rate	Step size shrinkage	0.3
use_label_encoder	Whether to use label encoder (set False to avoid warnings)	True
eval_metric	Metric to evaluate during training	Depends on task

Key Takeaways

Install and import XGBClassifier from the xgboost package before use.

Always set use_label_encoder=False and specify eval_metric to avoid warnings.

Fit the model with fit() on training data and predict with predict() on new data.

Tune parameters like n_estimators and max_depth to balance accuracy and overfitting.

Input data must be numeric; preprocess categorical features before training.