Elastic Net regularization in ML Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is Elastic Net regularization?

Elastic Net regularization is a technique that combines both L1 (Lasso) and L2 (Ridge) penalties to improve model performance and feature selection.

intermediate

Why do we use Elastic Net instead of just Lasso or Ridge?

Elastic Net is used because it balances the strengths of Lasso (feature selection) and Ridge (handling multicollinearity), making it effective when features are correlated.

beginner

What are the two penalty terms in Elastic Net?

Elastic Net uses a mix of L1 penalty (sum of absolute values of coefficients) and L2 penalty (sum of squared coefficients).

intermediate

How does Elastic Net help with correlated features?

Elastic Net tends to select groups of correlated features together, unlike Lasso which might pick only one, improving model stability.

advanced

Write the Elastic Net loss function formula.

The Elastic Net loss function is: <br>Loss = RSS + α * (λ * L1_penalty + (1 - λ) * L2_penalty), <br>where RSS is residual sum of squares, α controls overall strength, and λ balances L1 and L2 penalties.

What does the L1 penalty in Elastic Net encourage?

A Ignoring correlated features

B Shrinking coefficients towards zero but not zeroing them

C Sparsity by setting some coefficients to zero

D Increasing model complexity

L1 penalty encourages sparsity by pushing some coefficients exactly to zero, effectively selecting features.

Which problem does Elastic Net help solve better than Lasso alone?

A Handling correlated features

B Fitting non-linear models

C Reducing training time

D Increasing number of features

Elastic Net handles correlated features better by selecting groups of them together.

In Elastic Net, what does the parameter λ control?

A The learning rate

B The size of the dataset

C The number of features

D The balance between L1 and L2 penalties

λ controls how much weight is given to L1 versus L2 penalties in Elastic Net.

What happens if λ = 1 in Elastic Net?

A It becomes Lasso regression

B It becomes Ridge regression

C No regularization is applied

D Model overfits

When λ = 1, Elastic Net uses only the L1 penalty, which is Lasso regression.

Which of these is NOT a benefit of Elastic Net?

A Handling multicollinearity

B Guaranteeing zero training error

C Feature selection

D Improving model generalization

Elastic Net does not guarantee zero training error; it aims to reduce overfitting and improve generalization.

Explain in your own words how Elastic Net regularization works and why it is useful.

Think about how Lasso and Ridge work and how Elastic Net mixes them.

Describe the role of the parameters α and λ in Elastic Net regularization.

Consider how these parameters influence the penalty terms.