Feature importance explanation in ML Python

Introduction

Feature importance helps us understand which parts of our data matter most for making predictions. It shows what the model thinks is important.

You want to know which factors affect house prices the most.

You need to explain to a friend why a model made a certain decision.

You want to reduce the number of inputs to make a simpler model.

You are checking if the model is using sensible information.

You want to find out which features to focus on for improving data quality.

Syntax

ML Python

model.feature_importances_

This works for tree-based models like Random Forest or Gradient Boosting.

The output is an array showing importance scores for each feature.

Examples

Train a random forest and get feature importance scores.

ML Python

from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier()
model.fit(X_train, y_train)
importances = model.feature_importances_

Print each feature name with its importance score.

ML Python

importances = model.feature_importances_
for name, score in zip(feature_names, importances):
    print(f"{name}: {score:.3f}")

Sample Model

This code trains a random forest on the iris flower data and prints how important each feature is for predicting the flower type.

ML Python

from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier

# Load data
iris = load_iris()
X, y = iris.data, iris.target
feature_names = iris.feature_names

# Train model
model = RandomForestClassifier(random_state=42)
model.fit(X, y)

# Get feature importance
importances = model.feature_importances_

# Print results
for name, score in zip(feature_names, importances):
    print(f"{name}: {score:.3f}")

Important Notes

Feature importance values add up to 1.

Higher score means more influence on the model's decisions.

Not all models provide feature importance directly.

Summary

Feature importance shows which data features affect predictions most.

It helps explain and trust machine learning models.

Tree-based models like Random Forest provide easy access to these scores.