XGBoost in ML Python

Introduction

XGBoost helps us make smart predictions by learning from data quickly and accurately.

When you want to predict if an email is spam or not.

When you need to estimate house prices based on features like size and location.

When you want to classify images into categories.

When you want to improve prediction accuracy over simple models.

When you have structured data and want fast training with good results.

Syntax

ML Python

import xgboost as xgb
model = xgb.XGBClassifier()
model.fit(X_train, y_train)
predictions = model.predict(X_test)

XGBClassifier is used for classification tasks, while XGBRegressor is for regression.

You need to prepare your data as arrays or dataframes before training.

Examples

Create a classifier with max tree depth 3 and 100 trees.

ML Python

import xgboost as xgb
model = xgb.XGBClassifier(max_depth=3, n_estimators=100)
model.fit(X_train, y_train)

Create a regressor for predicting continuous values.

ML Python

model = xgb.XGBRegressor(objective='reg:squarederror')
model.fit(X_train, y_train)

Make predictions on test data and print first 5 results.

ML Python

preds = model.predict(X_test)
print(preds[:5])

Sample Model

This program trains an XGBoost model on iris flower data to classify species, then prints accuracy and first 5 predictions.

ML Python

import xgboost as xgb
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Load iris data
iris = load_iris()
X, y = iris.data, iris.target

# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Create XGBoost classifier
model = xgb.XGBClassifier(use_label_encoder=False, eval_metric='mlogloss')

# Train model
model.fit(X_train, y_train)

# Predict
predictions = model.predict(X_test)

# Calculate accuracy
acc = accuracy_score(y_test, predictions)
print(f"Accuracy: {acc:.2f}")
print(f"Predictions: {predictions[:5]}")

Important Notes

XGBoost is fast because it builds many small trees step-by-step.

You can tune parameters like tree depth and number of trees to improve results.

Always split your data into training and testing to check how well the model works.

Summary

XGBoost is a powerful tool for making predictions from data.

It works well on many types of problems like classification and regression.

You train it by giving data and labels, then use it to predict new data.