Regularization (Ridge, Lasso) in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Ridge vs Lasso Regularization

Which statement correctly describes the main difference between Ridge and Lasso regularization?

A Lasso regularization always performs better than Ridge because it uses both L1 and L2 penalties.

B Ridge adds an L1 penalty that sets coefficients to zero, while Lasso adds an L2 penalty that only shrinks coefficients.

C Both Ridge and Lasso add L2 penalties but differ in how they scale the penalty term.

D Ridge adds an L2 penalty which shrinks coefficients but does not set any to zero, while Lasso adds an L1 penalty that can set some coefficients exactly to zero.

💡 Hint

Think about which penalty type (L1 or L2) causes coefficients to become exactly zero.

Ridge uses L2 penalty which shrinks coefficients towards zero but does not make them exactly zero. Lasso uses L1 penalty which can shrink some coefficients exactly to zero, effectively performing feature selection.

❓ Predict Output

intermediate

Output of Lasso Regression Coefficients

What will be the output coefficients after fitting Lasso regression on the given data?

ML Python

from sklearn.linear_model import Lasso
import numpy as np

X = np.array([[1, 2], [2, 3], [3, 4], [4, 5]])
y = np.array([3, 5, 7, 9])

model = Lasso(alpha=1.0)
model.fit(X, y)
coefficients = model.coef_
print(coefficients)

A [0. 0.]

B [1. 1.]

C [0. 2.]

D [0.5 1.5]

💡 Hint

Recall that Lasso can zero out some coefficients depending on alpha.

With alpha=1.0, Lasso shrinks the first coefficient to zero and keeps the second close to 2, reflecting feature selection.

❓ Hyperparameter

advanced

Effect of Increasing Alpha in Ridge Regression

What happens to the coefficients of a Ridge regression model as the regularization parameter alpha increases?

A Coefficients remain unchanged regardless of alpha value.

B Coefficients shrink towards zero, reducing model complexity and overfitting.

C Coefficients become larger in magnitude to fit the training data better.

D Coefficients become exactly zero, performing feature selection.

💡 Hint

Think about how L2 penalty affects coefficient size as alpha grows.

Increasing alpha in Ridge regression increases the penalty on coefficient size, causing coefficients to shrink towards zero but not exactly zero.

❓ Metrics

advanced

Choosing Between Ridge and Lasso Using Cross-Validation

You have trained both Ridge and Lasso regression models on the same dataset using cross-validation. The Ridge model has a mean squared error (MSE) of 4.5, and the Lasso model has an MSE of 4.7. Which model should you choose and why?

A Choose Ridge because it has a lower MSE, indicating better predictive performance.

B Choose Lasso because it performs feature selection even if MSE is higher.

C Choose Lasso because it always generalizes better than Ridge.

D Choose Ridge only if the number of features is greater than the number of samples.

💡 Hint

Lower MSE means better prediction on unseen data.

Between two models, the one with lower cross-validation MSE is preferred as it predicts better on new data. Feature selection is useful but does not override predictive accuracy.

🔧 Debug

expert

Debugging Lasso Regression with Unexpected Zero Coefficients

You trained a Lasso regression model on your dataset, but all coefficients are zero. What is the most likely cause?

A The alpha parameter is set too high, causing all coefficients to shrink to zero.

B The dataset has too many features, causing Lasso to fail.

C The model was trained without fitting the data.

D The target variable is constant, so coefficients are zero.

💡 Hint

Think about how alpha controls the strength of regularization in Lasso.

If alpha is very large, Lasso penalizes coefficients heavily, shrinking them all to zero. This is a common cause of all-zero coefficients.

Practice

1. What is the main purpose of regularization in machine learning models?

easy

A. To prevent overfitting by adding a penalty to large weights

B. To increase the size of the training dataset

C. To speed up the training process by reducing data

D. To remove all features from the model

Solution

1. Step 1: Understand overfitting

   Overfitting happens when a model learns noise instead of the true pattern, causing poor performance on new data.

2. Step 2: Role of regularization

   Regularization adds a penalty to large weights, which helps the model generalize better by keeping weights small.

3. Final Answer:

   To prevent overfitting by adding a penalty to large weights -> Option A

4. Quick Check:

   Regularization = Prevent overfitting [OK]

Hint: Regularization controls model complexity by penalizing big weights [OK]

Common Mistakes:

• Thinking regularization increases data size
• Confusing regularization with feature removal
• Believing regularization speeds up training

2. Which of the following is the correct way to import Ridge regression from scikit-learn in Python?

easy

A. import sklearn.linear_model.Ridge

B. from sklearn.linear_model import Ridge

C. from sklearn import Ridge

D. import Ridge from sklearn.linear_model

Solution

1. Step 1: Recall Python import syntax

   In Python, to import a class from a module, the syntax is 'from module import Class'.

2. Step 2: Apply to Ridge regression

   Ridge is a class inside sklearn.linear_model, so the correct import is 'from sklearn.linear_model import Ridge'.

3. Final Answer:

   from sklearn.linear_model import Ridge -> Option B

4. Quick Check:

   Correct import syntax = from sklearn.linear_model import Ridge [OK]

Hint: Use 'from module import Class' to import classes in Python [OK]

Common Mistakes:

• Using 'import Class from module' which is invalid syntax
• Trying to import module and class together incorrectly
• Using dot notation in import statement

3. Consider the following Python code using Lasso regression:

from sklearn.linear_model import Lasso
import numpy as np

X = np.array([[1, 2], [3, 4], [5, 6]])
y = np.array([1, 2, 3])
model = Lasso(alpha=2.0)
model.fit(X, y)
pred = model.predict(np.array([[7, 8]]))
print(round(pred[0], 2))

What is the printed output?

medium

A. 2.5

B. 2.0

C. 3.0

D. 1.0

Solution

1. Step 1: Understand Lasso fitting on small data

   Lasso applies L1 penalty, shrinking some coefficients possibly to zero. With alpha=2.0, coefficients shrink significantly.

2. Step 2: Predict value for input [7, 8]

   Given the linear trend in X and y (unregularized prediction 4.0), with alpha=2.0 the prediction is approximately 2.5 after regularization.

3. Final Answer:

   2.5 -> Option A

4. Quick Check:

   Lasso prediction ≈ 2.5 [OK]

Hint: Lasso shrinks coefficients, prediction near linear trend but reduced by alpha [OK]

Common Mistakes:

• Assuming prediction equals last y value
• Ignoring effect of alpha on coefficients
• Confusing Ridge and Lasso penalties

4. The following code tries to use Ridge regression but throws an error:

from sklearn.linear_model import Ridge
model = Ridge(alpha='one')
model.fit(X_train, y_train)

What is the error and how to fix it?

medium

A. SyntaxError due to wrong import statement; fix import syntax

B. NameError because Ridge is not imported; fix by importing Ridge

C. ValueError because X_train is missing; fix by defining X_train

D. ValueError because alpha must be a float; fix by using alpha=1.0

Solution

1. Step 1: Identify the error cause

   Alpha parameter expects a numeric value (float), but 'one' is a string causing ValueError.

2. Step 2: Fix the error

   Replace alpha='one' with alpha=1.0 to provide a valid float value.

3. Final Answer:

   ValueError because alpha must be a float; fix by using alpha=1.0 -> Option D

4. Quick Check:

   Alpha must be float, not string [OK]

Hint: Alpha must be a number, not a string [OK]

Common Mistakes:

• Passing string instead of float for alpha
• Ignoring error message details
• Assuming import error without checking code

5. You have a dataset with many features, some of which are irrelevant. You want a model that both reduces overfitting and selects important features by setting some coefficients exactly to zero. Which regularization method should you choose?

hard

A. Ridge regression because it shrinks coefficients but keeps all features

B. Neither Ridge nor Lasso; use standard linear regression

C. Lasso regression because it can set some coefficients to zero, removing features

D. Use Ridge with very small alpha to remove features

Solution

1. Step 1: Understand feature selection need

   To remove irrelevant features, the model must set some coefficients exactly to zero.

2. Step 2: Compare Ridge and Lasso

   Ridge shrinks coefficients but does not zero them out; Lasso uses L1 penalty and can zero coefficients, effectively selecting features.

3. Final Answer:

   Lasso regression because it can set some coefficients to zero, removing features -> Option C

4. Quick Check:

   Lasso = feature selection + regularization [OK]

Hint: Lasso zeros coefficients; Ridge only shrinks [OK]

Common Mistakes:

• Thinking Ridge removes features by zeroing coefficients
• Choosing no regularization for feature selection
• Confusing alpha size effect on feature removal