Random forest in depth in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

How does a random forest reduce overfitting compared to a single decision tree?

Random forests build many decision trees and combine their results. Why does this help reduce overfitting compared to using just one tree?

A Because averaging many trees reduces variance and prevents the model from fitting noise in the training data.

B Because random forests use deeper trees that memorize the training data better.

C Because random forests only use a single feature for all splits, making the model simpler.

D Because random forests remove all noisy data points before training.

💡 Hint

Think about how averaging multiple guesses can make the final guess more stable.

Random forests reduce overfitting by averaging many trees trained on different random subsets of data and features. This averaging lowers variance and prevents the model from fitting noise.

❓ Predict Output

intermediate

Output of feature importance extraction from a random forest

What is the output of this code snippet that trains a random forest and prints feature importances?

ML Python

from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier

iris = load_iris()
X, y = iris.data, iris.target
model = RandomForestClassifier(n_estimators=10, random_state=42)
model.fit(X, y)
importances = model.feature_importances_
print([round(i, 2) for i in importances])

A [0.5, 0.2, 0.2, 0.1]

B [0.25, 0.25, 0.25, 0.25]

C [0.11, 0.03, 0.44, 0.42]

D [0.0, 0.0, 0.0, 1.0]

💡 Hint

Feature importances sum to 1 and reflect how useful each feature is for splitting.

The printed list shows the importance of each feature rounded to two decimals. The last two features have higher importance in this trained model.

❓ Hyperparameter

advanced

Which hyperparameter controls the number of features considered at each split in a random forest?

When training a random forest, which hyperparameter decides how many features are randomly selected to consider for splitting at each node?

A max_depth

B max_features

C min_samples_split

D n_estimators

💡 Hint

This parameter controls randomness in feature selection per split.

max_features controls how many features are randomly chosen at each split, adding randomness and diversity to trees.

❓ Metrics

advanced

Interpreting out-of-bag (OOB) error in random forests

What does the out-of-bag (OOB) error estimate in a random forest represent?

A The error on data not used to train each tree, providing an unbiased estimate of test error.

B The error on the training data used to build each tree.

C The error on a separate validation dataset provided by the user.

D The error after pruning the trees to reduce complexity.

💡 Hint

OOB samples are those left out when bootstrapping data for each tree.

OOB error is calculated using samples not included in the bootstrap sample for each tree, giving an unbiased estimate of model performance without needing a separate validation set.

🔧 Debug

expert

Why does this random forest model raise a ValueError?

Consider this code snippet:

from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators=100, max_features='auto')
model.fit(X_train, y_train)

Why does this code raise a ValueError?

A Because RandomForestClassifier requires max_depth to be set explicitly.

B Because n_estimators must be less than 50.

C Because X_train and y_train are not defined.

D Because 'auto' is not a valid value for max_features in RandomForestClassifier in recent sklearn versions.

💡 Hint

Check the allowed values for max_features in sklearn 1.2+.

In recent sklearn versions, 'auto' is deprecated for max_features in RandomForestClassifier. Use 'sqrt' instead.

Practice

1. What is the main advantage of using a random forest over a single decision tree?

easy

A. It reduces overfitting by averaging multiple trees.

B. It always runs faster than a single tree.

C. It requires less data to train.

D. It uses only one feature for splitting.

Solution

1. Step 1: Understand decision tree limitations

   A single decision tree can easily overfit, meaning it learns noise and performs poorly on new data.

2. Step 2: How random forest improves

   Random forest builds many trees on random subsets of data and features, then averages their results to reduce overfitting.

3. Final Answer:

   It reduces overfitting by averaging multiple trees. -> Option A

4. Quick Check:

   Random forest reduces overfitting = B [OK]

Hint: Random forest averages trees to avoid overfitting [OK]

Common Mistakes:

• Thinking random forest is always faster than one tree
• Believing it uses fewer data than a single tree
• Assuming it splits on only one feature

2. Which of the following is the correct way to create a random forest classifier in Python using scikit-learn?

easy

A. from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators=100)

B. from sklearn.tree import RandomForest
model = RandomForest(100)

C. import randomforest
model = randomforest.RandomForestClassifier(100)

D. from sklearn.ensemble import RandomForest
model = RandomForest(n_trees=100)

Solution

1. Step 1: Identify correct import

   The random forest classifier is in sklearn.ensemble as RandomForestClassifier.

2. Step 2: Check constructor usage

   We create it by calling RandomForestClassifier with n_estimators=100 to set number of trees.

3. Final Answer:

   from sklearn.ensemble import RandomForestClassifier model = RandomForestClassifier(n_estimators=100) -> Option A

4. Quick Check:

   Correct import and parameter = A [OK]

Hint: Use sklearn.ensemble.RandomForestClassifier with n_estimators [OK]

Common Mistakes:

• Importing from sklearn.tree instead of sklearn.ensemble
• Using wrong class names like RandomForest
• Passing wrong parameter names like n_trees

3. Consider this Python code using scikit-learn's random forest:

from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators=3, max_depth=2, random_state=42)
X = [[0, 0], [1, 1], [0, 1], [1, 0]]
y = [0, 1, 1, 0]
model.fit(X, y)
preds = model.predict([[0, 0], [1, 1]])
print(list(preds))

What is the output?

medium

A. [0, 0]

B. [0, 1]

C. [1, 0]

D. [1, 1]

Solution

1. Step 1: Understand training data and labels

   Input points [0,0] and [1,1] have labels 0 and 1 respectively.

2. Step 2: Predict on same points with trained model

   Random forest with 3 trees and max depth 2 will learn simple splits and predict correctly on these points.

3. Final Answer:

   [0, 1] -> Option B

4. Quick Check:

   Predictions match training labels = C [OK]

Hint: Predictions on training points usually match labels [OK]

Common Mistakes:

• Confusing input order and labels
• Assuming random forest predicts opposite labels
• Ignoring max_depth effect

4. You wrote this code but get an error:

from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators='100')
model.fit(X_train, y_train)

What is the problem?

medium

A. fit method requires extra parameters.

B. RandomForestClassifier does not have n_estimators parameter.

C. n_estimators should be an integer, not a string.

D. You must import RandomForestRegressor instead.

Solution

1. Step 1: Check parameter type for n_estimators

   n_estimators expects an integer number of trees, not a string.

2. Step 2: Identify error cause

   Passing '100' as a string causes a type error during model creation or training.

3. Final Answer:

   n_estimators should be an integer, not a string. -> Option C

4. Quick Check:

   Parameter type mismatch = A [OK]

Hint: Use integer for n_estimators, not string [OK]

Common Mistakes:

• Passing numbers as strings in parameters
• Confusing classifier and regressor classes
• Thinking fit needs extra arguments

5. You want to improve your random forest model's accuracy on a complex dataset. Which combination of hyperparameters is best to try first?

hard

A. Set max_depth to 1 and keep n_estimators low

B. Decrease n_estimators and decrease max_depth

C. Increase max_features to total features and decrease n_estimators

D. Increase n_estimators and increase max_depth

Solution

1. Step 1: Understand effect of n_estimators

   More trees (higher n_estimators) usually improve accuracy by reducing variance.

2. Step 2: Understand effect of max_depth

   Increasing max_depth allows trees to learn more complex patterns, improving accuracy on complex data.

3. Final Answer:

   Increase n_estimators and increase max_depth -> Option D

4. Quick Check:

   More trees + deeper trees = better accuracy [OK]

Hint: More trees and deeper trees usually improve accuracy [OK]

Common Mistakes:

• Reducing trees and depth lowers accuracy
• Setting max_depth too low causes underfitting
• Increasing max_features too much can cause overfitting