Recursive feature elimination in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the main goal of Recursive Feature Elimination (RFE)?

Imagine you have many features in your dataset. What does RFE try to do with these features?

A It tries to select the most important features by removing the least important ones step by step.

B It creates new features by combining existing ones to improve model accuracy.

C It randomly removes features to reduce dataset size without considering importance.

D It increases the number of features by duplicating existing ones to add more data.

💡 Hint

Think about how RFE helps simplify the model by focusing on key features.

RFE works by repeatedly training a model and removing the weakest features until the desired number remains. This helps keep only the most useful features.

❓ Predict Output

intermediate

What is the output shape of X after applying RFE with 3 features?

Given the code below, what will be the shape of X_rfe?

ML Python

from sklearn.datasets import load_iris
from sklearn.feature_selection import RFE
from sklearn.linear_model import LogisticRegression

iris = load_iris()
X = iris.data
Y = iris.target

model = LogisticRegression(max_iter=200)
rfe = RFE(model, n_features_to_select=3)
rfe = rfe.fit(X, Y)
X_rfe = rfe.transform(X)
print(X_rfe.shape)

A (4, 150)

B (150, 4)

C (3, 150)

D (150, 3)

💡 Hint

RFE reduces the number of features but keeps the number of samples the same.

The original data has 150 samples and 4 features. After RFE selects 3 features, the shape becomes (150, 3).

❓ Model Choice

advanced

Which model is best suited for RFE in this scenario?

You want to use RFE to select features for a classification task with a small dataset. Which model below is most appropriate to use with RFE?

A Logistic Regression with L2 regularization

B K-Nearest Neighbors without feature importance

C K-Means clustering

D Principal Component Analysis (PCA)

💡 Hint

RFE needs a model that can provide feature importance or coefficients.

Logistic Regression provides coefficients that RFE uses to rank features. KNN and K-Means do not provide feature importance, and PCA is a dimensionality reduction method, not a model for RFE.

❓ Hyperparameter

advanced

What does the hyperparameter n_features_to_select control in RFE?

In Recursive Feature Elimination, what is the role of n_features_to_select?

A It specifies the percentage of features to remove at each step.

B It sets the number of features to keep after elimination.

C It controls the number of iterations to run the model.

D It defines the minimum importance score for features to be kept.

💡 Hint

Think about how many features you want to end up with after RFE finishes.

The n_features_to_select parameter tells RFE how many features to keep after the elimination process.

❓ Metrics

expert

How to evaluate if RFE improved model performance?

You applied RFE to select features and trained a model. Which metric comparison best shows if RFE helped?

A Compare the time taken to train the model without checking accuracy.

B Compare training loss only on the training data before and after RFE.

C Compare model accuracy on a test set before and after RFE.

D Compare the number of features selected without checking model results.

💡 Hint

Think about how to know if the model got better at predicting new data.

Comparing accuracy on a test set before and after RFE shows if the feature selection improved the model's ability to predict unseen data.