Recursive feature elimination in ML Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is Recursive Feature Elimination (RFE)?

RFE is a method to select important features by repeatedly training a model, ranking features by importance, and removing the least important ones until the best set remains.

beginner

Why do we use Recursive Feature Elimination?

We use RFE to improve model performance and reduce complexity by keeping only the most useful features and removing irrelevant or noisy ones.

intermediate

How does RFE decide which features to remove?

RFE trains a model and ranks features by their importance scores (like coefficients or feature importances). It removes the least important features in each step.

intermediate

What types of models can be used with RFE?

RFE works with models that provide feature importance, such as linear models with coefficients or tree-based models with feature importance scores.

intermediate

What is a common stopping criterion in RFE?

RFE stops when a desired number of features is reached or when removing more features hurts model performance.

What does Recursive Feature Elimination do?

A Normalizes all features

B Adds new features to the dataset

C Randomly selects features

D Removes features one by one based on importance

RFE removes the least important features step-by-step based on model importance scores.

Which model type is suitable for RFE?

A Models that provide feature importance

B Models that only predict labels

C Models without coefficients

D Models that do not train

RFE requires models that can rank features by importance, like linear or tree-based models.

When does RFE usually stop removing features?

A When all features are removed

B After one iteration

C When the desired number of features is reached

D When accuracy drops to zero

RFE stops when it reaches the target number of features or performance worsens.

What is the main goal of using RFE?

A To increase dataset size

B To select the most important features

C To create new features

D To shuffle data randomly

RFE aims to keep only the most useful features for better model performance.

Which of these is NOT a step in RFE?

A Add random noise to features

B Rank features by importance

C Remove least important features

D Train model on current features

RFE does not add noise; it removes features based on importance.

Explain how Recursive Feature Elimination works step-by-step.

Think about training, ranking, removing, and repeating.

Why is feature selection important and how does RFE help with it?

Consider benefits of fewer features and how RFE chooses them.