Bias-variance tradeoff in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Bias and Variance

Which statement best describes the relationship between bias and variance in a machine learning model?

A Low bias always leads to low variance in a model.

B High bias means the model fits the training data perfectly, while high variance means the model cannot fit the training data well.

C Bias and variance both measure how well the model performs on the training data only.

D High bias means the model is too simple and underfits, while high variance means the model is too complex and overfits.

❓ Predict Output

intermediate

Effect of Model Complexity on Error

Consider the following Python code that simulates training and test errors for models of increasing complexity. What is the output of the print statement?

ML Python

import numpy as np
complexity = np.array([1, 2, 3, 4, 5])
train_error = 1 / complexity
test_error = 1 / complexity + 0.1 * complexity
print(f"Train error: {train_error}")
print(f"Test error: {test_error}")

A Train error decreases as complexity increases; test error decreases then increases.

B Both train and test errors increase as complexity increases.

C Train error increases as complexity increases; test error decreases as complexity increases.

D Train error stays constant; test error decreases as complexity increases.

❓ Hyperparameter

advanced

Choosing Regularization Strength

You train a linear regression model with L2 regularization (Ridge). Increasing the regularization parameter lambda will most likely:

A Decrease bias and increase variance, leading to more complex models.

B Increase bias and decrease variance, leading to simpler models.

C Increase both bias and variance, making the model unstable.

D Have no effect on bias or variance.

❓ Metrics

advanced

Interpreting Learning Curves

You observe the following behavior in learning curves: training error is high and stable, but test error is high and does not improve with more data. What does this indicate about the model?

A The model has data leakage causing artificially low training error.

B The model has high variance and is overfitting the training data.

C The model has high bias and is underfitting the data.

D The model is perfectly balanced between bias and variance.

🔧 Debug

expert

Diagnosing Model Behavior from Code

Given the following code snippet training a decision tree, what is the most likely cause of the model having high training error and high test error?

from sklearn.tree import DecisionTreeClassifier
model = DecisionTreeClassifier(max_depth=1)
model.fit(X_train, y_train)
train_acc = model.score(X_train, y_train)
test_acc = model.score(X_test, y_test)
print(f"Train accuracy: {train_acc}")
print(f"Test accuracy: {test_acc}")

A The max_depth=1 is too low, causing high bias and underfitting.

B The max_depth=1 is too high, causing high variance and overfitting.

C The model is overfitting because max_depth is not set, so it grows fully.

D The training data is too small, causing unstable accuracy scores.