Feature selection methods in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Which feature selection method uses model coefficients to select important features?

Imagine you have a dataset with many features. You want to pick the most important ones by looking at the model's learned weights. Which feature selection method does this?

A Filter method using correlation coefficients

B Wrapper method using recursive feature elimination

C Dimensionality reduction using PCA

D Embedded method using Lasso regression coefficients

💡 Hint

Think about methods that select features during model training by shrinking some coefficients to zero.

Embedded methods like Lasso regression include feature selection as part of the model training by penalizing coefficients. This causes some coefficients to become zero, effectively selecting features.

❓ Predict Output

intermediate

What is the number of features selected by this code snippet?

Consider the following Python code using sklearn to select features based on univariate statistical tests.

ML Python

from sklearn.datasets import load_iris
from sklearn.feature_selection import SelectKBest, f_classif

X, y = load_iris(return_X_y=True)
selector = SelectKBest(score_func=f_classif, k=2)
X_new = selector.fit_transform(X, y)
num_features = X_new.shape[1]

A 3

B 4

C 2

D 1

💡 Hint

Look at the parameter k in SelectKBest.

The SelectKBest method selects exactly k features. Here, k=2, so the output has 2 features.

❓ Model Choice

advanced

Which model is best suited for embedded feature selection in high-dimensional sparse data?

You have a dataset with thousands of features but only a few samples. You want a model that can select important features while training. Which model is best?

A Lasso Regression

B Random Forest Classifier

C K-Nearest Neighbors

D Support Vector Machine with RBF kernel

💡 Hint

Think about models that add penalties to reduce coefficients to zero.

Lasso regression adds an L1 penalty that forces many coefficients to zero, making it suitable for feature selection in high-dimensional sparse data.

❓ Hyperparameter

advanced

What effect does increasing the 'k' parameter in SelectKBest have?

In the SelectKBest feature selection method, what happens if you increase the value of k?

A More features are selected, possibly including less relevant ones

B Fewer features are selected, focusing only on the top ones

C The model automatically tunes <code>k</code> during training

D The method switches from filter to wrapper approach

💡 Hint

Think about what k controls in feature selection.

The parameter k controls how many top features are selected. Increasing k means selecting more features, which may include less important ones.

❓ Metrics

expert

Which metric best evaluates feature selection effectiveness in classification?

You want to measure how well your feature selection improved your classification model. Which metric is most appropriate to compare before and after feature selection?

A Mean squared error on training data

B Model accuracy on a validation set

C Number of features selected

D Training time of the model

💡 Hint

Think about what shows if the model predicts better with selected features.

Model accuracy on a validation set shows if feature selection improved the model's ability to predict unseen data.