Bagging concept in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the main purpose of bagging in machine learning?

Bagging is a technique used in machine learning. What is its main goal?

A To increase the bias of a model by simplifying the training data.

B To reduce the size of the training dataset to speed up training.

C To combine models by selecting the single best performing model.

D To reduce the variance of a model by training multiple models on different samples and averaging their predictions.

💡 Hint

Think about how bagging uses multiple models and what problem it tries to solve.

Bagging stands for Bootstrap Aggregating. It reduces variance by training multiple models on different random samples of the data and then averaging their predictions to get a more stable result.

❓ Predict Output

intermediate

Output of bagging predictions averaging

Given three models trained on different samples, their predictions on a test point are: Model1: 0.7, Model2: 0.4, Model3: 0.9. What is the final bagging prediction by averaging?

A 0.73

B 0.67

C 0.80

D 0.50

💡 Hint

Calculate the average of the three predictions.

The average is (0.7 + 0.4 + 0.9) / 3 = 2.0 / 3 = 0.666..., rounded to 0.67.

❓ Model Choice

advanced

Which model type benefits most from bagging?

Bagging is most effective in reducing variance. Which of these model types typically benefits the most from bagging?

A Decision trees with high depth (complex trees)

B Linear regression models

C Simple logistic regression models

D Naive Bayes classifiers

💡 Hint

Think about which models tend to have high variance and overfit easily.

Complex decision trees tend to overfit and have high variance. Bagging helps by averaging many trees trained on different samples, reducing variance and improving stability.

❓ Hyperparameter

advanced

Effect of increasing number of base models in bagging

What is the effect of increasing the number of base models (estimators) in a bagging ensemble?

A It generally decreases variance and improves stability up to a point, but with diminishing returns.

B It increases bias and reduces model accuracy.

C It causes the model to overfit the training data more.

D It has no effect on the ensemble's performance.

💡 Hint

Think about how averaging more models affects variance and prediction stability.

Adding more base models reduces variance by averaging more predictions, improving stability. However, after many models, improvements become smaller.

🔧 Debug

expert

Identify the error in this bagging implementation snippet

Consider this Python code snippet for bagging:

from sklearn.tree import DecisionTreeClassifier
from sklearn.utils import resample

X_train, y_train = ...  # training data
models = []
for _ in range(5):
    X_sample, y_sample = resample(X_train, y_train)
    model = DecisionTreeClassifier()
    model.fit(X_sample, y_sample)
    models.append(model)

# Predict on test data
predictions = []
for model in models:
    predictions.append(model.predict(X_test))

final_prediction = sum(predictions) / len(models)

What error will this code raise or what is the problem?

A NameError because X_test is not defined.

B ValueError because resample requires additional parameters.

C TypeError because predictions are arrays and cannot be summed directly with sum().

D No error; code runs correctly and outputs final predictions.

💡 Hint

Check the type of objects in predictions and how sum() works on lists of arrays.

Each model.predict returns a numpy array. Using sum() on a list of arrays tries to add arrays starting from 0 (int), causing a TypeError. Instead, use numpy.sum or initialize with zeros array.