Why ensembles outperform single models in ML Python - Challenge Your Understanding

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why do ensemble models usually perform better than single models?

Imagine you ask several friends for advice instead of just one. Why might their combined advice be better? Similarly, why do ensemble models often outperform a single model?

A Because ensembles ignore the predictions of weaker models.

B Because ensembles always use deeper neural networks than single models.

C Because ensembles train on more data than single models.

D Because ensembles combine multiple models, reducing errors from individual models and improving overall accuracy.

💡 Hint

Think about how combining different opinions can reduce mistakes.

Ensembles combine predictions from multiple models, which helps cancel out individual errors and leads to better accuracy. They do not necessarily use deeper networks or more data, and they do not ignore weaker models but rather weigh all predictions.

❓ Metrics

intermediate

Effect of ensemble on model variance

You have 5 models each with variance 0.04 and zero covariance between them. What is the variance of the average prediction from these 5 models?

A 0.008

B 0.04

C 0.2

D 0.0008

💡 Hint

Variance of average of independent variables is variance divided by number of variables.

Variance of average = variance / number of models = 0.04 / 5 = 0.008.

❓ Predict Output

advanced

Output of ensemble prediction averaging

What is the output of the following Python code that averages predictions from three models?

ML Python

predictions = [[0.2, 0.8], [0.3, 0.7], [0.1, 0.9]]
avg_pred = [sum(x)/len(x) for x in zip(*predictions)]
print([round(p, 2) for p in avg_pred])

A [0.3, 0.7]

B [0.15, 0.85]

C [0.2, 0.8]

D [0.5, 0.5]

💡 Hint

Calculate average for each class probability across models.

For class 0: (0.2+0.3+0.1)/3=0.2; for class 1: (0.8+0.7+0.9)/3=0.8.

❓ Model Choice

advanced

Choosing ensemble type for reducing bias and variance

You want to reduce both bias and variance in your model predictions. Which ensemble method is best suited for this?

A Simple averaging of identical models

B Bagging (e.g., Random Forest)

C Boosting (e.g., Gradient Boosting Machines)

D Using a single deep neural network

💡 Hint

Think about which method focuses on correcting errors and improving weak learners.

Boosting reduces bias by sequentially correcting errors and also reduces variance by combining models. Bagging mainly reduces variance. Simple averaging identical models does not reduce bias. Single deep networks may have high variance or bias.

🔧 Debug

expert

Why does this ensemble code produce wrong predictions?

Consider this code that tries to ensemble predictions by majority vote. What is the bug causing incorrect output?

ML Python

import numpy as np
preds = [[1,0,1],[0,1,1],[1,1,0]]
ensemble_pred = [np.argmax(np.bincount(preds)) for i in range(len(preds[0]))]
print(ensemble_pred)

A np.bincount is called on the whole list instead of per position, causing wrong counts.

B The list comprehension incorrectly uses i in range(len(preds[0])) causing index errors.

C np.bincount is called on a list of predictions per position correctly; no bug here.

D np.argmax is used incorrectly; it should be np.argmin to get majority vote.

💡 Hint

Check what data np.bincount receives inside the list comprehension.

The code calls np.bincount on the entire list of predictions instead of the list of predictions at each position, so counts are wrong and majority vote fails.