Feature importance explanation in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding feature importance in decision trees

Which statement best describes how feature importance is calculated in a decision tree model?

A It counts how many times a feature appears in the dataset.

B It measures the average value of each feature across all samples.

C It calculates the correlation between each feature and the target variable.

D It measures how much each feature decreases the weighted impurity in the tree's splits.

💡 Hint

Think about how decision trees decide where to split data.

Decision trees calculate feature importance by summing how much each feature reduces impurity (like Gini or entropy) when used for splits, weighted by the number of samples it splits.

❓ Predict Output

intermediate

Output of feature importance extraction code

What is the output of the following Python code using scikit-learn's RandomForestClassifier?

ML Python

from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris

iris = load_iris()
X, y = iris.data, iris.target
model = RandomForestClassifier(random_state=0)
model.fit(X, y)
importances = model.feature_importances_
print([round(i, 2) for i in importances])

A [0.11, 0.03, 0.44, 0.42]

B [0.25, 0.25, 0.25, 0.25]

C [0.0, 0.0, 0.0, 1.0]

D [0.5, 0.2, 0.2, 0.1]

💡 Hint

Random forests usually assign higher importance to features that better split the data.

The output shows the relative importance of each feature in the iris dataset as computed by the random forest. Sepal length and width have lower importance compared to petal length and petal width.

❓ Hyperparameter

advanced

Effect of max_features on feature importance in Random Forest

How does setting the max_features parameter to a low value in a Random Forest affect the computed feature importance?

A It has no effect on feature importance values.

B It increases the importance of all features equally.

C It can reduce the importance of some features because fewer features are considered at each split.

D It causes the model to ignore the target variable.

💡 Hint

Think about how limiting features at splits changes the model's view of data.

When max_features is low, each tree sees fewer features at each split, so some features may be less likely to be chosen, lowering their importance.

❓ Metrics

advanced

Interpreting SHAP values for feature importance

Which statement correctly describes what SHAP values represent in feature importance analysis?

A SHAP values are random numbers assigned to features for visualization.

B SHAP values show how much each feature contributes to pushing a prediction from the average prediction to the actual prediction.

C SHAP values measure the correlation between features and the target variable.

D SHAP values count how many times a feature appears in the training data.

💡 Hint

SHAP values explain individual predictions by comparing to a baseline.

SHAP values quantify each feature's contribution to the difference between the model's average output and the specific prediction, helping explain model decisions.

🔧 Debug

expert

Identifying error in feature importance extraction code

What error will the following code raise when trying to get feature importances from a trained model?

ML Python

from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_iris

iris = load_iris()
X, y = iris.data, iris.target
model = LogisticRegression(max_iter=200)
model.fit(X, y)
print(model.feature_importances_)

A AttributeError: 'LogisticRegression' object has no attribute 'feature_importances_'

B ValueError: Invalid parameter 'feature_importances_'

C TypeError: 'feature_importances_' is not callable

D No error; prints feature importances as a list

💡 Hint

Not all models provide feature importance attributes.

LogisticRegression does not have a feature_importances_ attribute. This attribute exists in tree-based models like RandomForest but not in linear models.