Random forest classifier in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

How does a random forest reduce overfitting compared to a single decision tree?

Random forests use many decision trees to make predictions. Which of the following best explains how this helps reduce overfitting?

A By pruning each tree to a single node, it simplifies the model and prevents overfitting.

B By using only one tree with the deepest splits, it captures all data patterns perfectly, avoiding overfitting.

C By training all trees on the same data and features, it ensures consistent predictions and reduces overfitting.

D By averaging predictions from many trees trained on different random subsets of data and features, it reduces variance and overfitting.

💡 Hint

Think about how randomness and averaging help in machine learning models.

Random forests build many trees on random subsets of data and features. Averaging their predictions lowers variance and reduces overfitting compared to a single tree.

❓ Predict Output

intermediate

Output of random forest prediction probabilities

What is the output of the following Python code using scikit-learn's RandomForestClassifier?

ML Python

from sklearn.ensemble import RandomForestClassifier
import numpy as np

X_train = np.array([[1, 2], [3, 4], [5, 6], [7, 8]])
y_train = np.array([0, 1, 0, 1])
model = RandomForestClassifier(random_state=42)
model.fit(X_train, y_train)

X_test = np.array([[2, 3]])
pred_probs = model.predict_proba(X_test)
print(pred_probs)

A [[0.5 0.5]]

B [[0. 1.]]

C [[1. 0.]]

D [[0.75 0.25]]

💡 Hint

Consider how the model predicts class probabilities based on training data and the test input.

The model predicts class 1 with full confidence for input [2, 3] because the trees vote mostly for class 1 based on training data.

❓ Hyperparameter

advanced

Effect of increasing n_estimators in RandomForestClassifier

What is the most likely effect of increasing the n_estimators parameter (number of trees) in a RandomForestClassifier?

A It generally improves model stability and accuracy but increases training time.

B It has no effect on model performance or training time.

C It reduces training time by using fewer trees to make predictions.

D It decreases model accuracy because too many trees cause overfitting.

💡 Hint

Think about how more trees affect ensemble models.

More trees usually improve accuracy and stability by averaging more predictions but require more time to train.

❓ Metrics

advanced

Choosing the best metric for imbalanced classification with Random Forest

You train a RandomForestClassifier on a dataset where 95% of samples belong to class 0 and 5% to class 1. Which metric is best to evaluate your model's performance?

A F1-score, because it balances precision and recall for imbalanced data.

B Precision, because it measures how many predicted positives are correct.

C Recall, because it measures how many actual positives are found.

D Accuracy, because it shows overall correct predictions.

💡 Hint

Think about why accuracy can be misleading with imbalanced classes.

F1-score balances precision and recall, making it better for imbalanced datasets than accuracy alone.

🔧 Debug

expert

Debugging RandomForestClassifier training error

What error will this code raise when training a RandomForestClassifier, and why?

ML Python

from sklearn.ensemble import RandomForestClassifier

X_train = [[1, 2], [3, 4], [5, 6]]
y_train = [0, 1]
model = RandomForestClassifier()
model.fit(X_train, y_train)

A AttributeError: 'RandomForestClassifier' object has no attribute 'fit'

B TypeError: 'list' object is not callable

C ValueError: Found input variables with inconsistent numbers of samples

D No error, model trains successfully

💡 Hint

Check if the number of samples in features and labels match.

The feature list has 3 samples but the label list has only 2, causing a ValueError about inconsistent sample sizes.

Practice

1. What is the main idea behind a Random Forest Classifier?

easy

A. It clusters data without labels.

B. It uses only one decision tree for prediction.

C. It is a type of neural network.

D. It combines many decision trees to make better predictions.

Solution

1. Step 1: Understand what a random forest is

   A random forest is a group of decision trees working together.

2. Step 2: Identify the main purpose

   Combining many trees helps improve prediction accuracy and reduce errors.

3. Final Answer:

   It combines many decision trees to make better predictions. -> Option D

4. Quick Check:

   Random forest = many trees combined [OK]

Hint: Random forest means many trees working together [OK]

Common Mistakes:

• Thinking it uses only one tree
• Confusing it with neural networks
• Believing it does clustering without labels

2. Which of the following is the correct way to import and create a random forest classifier in Python using scikit-learn?

easy

A. from sklearn.tree import RandomForest
model = RandomForest(100)

B. import randomforest
model = randomforest.RandomForest(100)

C. from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators=100)

D. from sklearn.ensemble import RandomForest
model = RandomForestClassifier()

Solution

1. Step 1: Check the correct import

   The random forest classifier is in sklearn.ensemble as RandomForestClassifier.

2. Step 2: Verify the model creation syntax

   Use RandomForestClassifier(n_estimators=100) to create with 100 trees.

3. Final Answer:

   from sklearn.ensemble import RandomForestClassifier model = RandomForestClassifier(n_estimators=100) -> Option C

4. Quick Check:

   Correct import and model creation [OK]

Hint: Use sklearn.ensemble and RandomForestClassifier [OK]

Common Mistakes:

• Importing from sklearn.tree instead of ensemble
• Using wrong class names
• Missing n_estimators parameter

3. What will be the output of the following code snippet?

from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, random_state=42)
model = RandomForestClassifier(n_estimators=10, random_state=42)
model.fit(X_train, y_train)
predictions = model.predict(X_test)
print(list(predictions[:5]))

medium

A. [2, 2, 1, 0, 0]

B. [0, 1, 2, 0, 1]

C. [1, 0, 2, 1, 1]

D. [1, 1, 1, 1, 1]

Solution

1. Step 1: Understand the dataset and split

   The iris dataset has 3 classes (0,1,2). The split is random_state=42 for reproducibility.

2. Step 2: Fit and predict with random forest

   Using 10 trees and random_state=42, predictions on test data are consistent.

3. Final Answer:

   [2, 2, 1, 0, 0] -> Option A

4. Quick Check:

   Predictions match expected output with random_state=42 [OK]

Hint: Set random_state for reproducible predictions [OK]

Common Mistakes:

• Ignoring random_state causing different outputs
• Confusing class labels order
• Assuming all predictions are the same

4. Identify the error in the following code snippet that tries to train a random forest classifier:

from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_trees=100)
model.fit(X_train, y_train)

medium

A. The parameter name should be n_estimators, not n_trees.

B. RandomForestClassifier does not have a fit method.

C. X_train and y_train must be numpy arrays, not lists.

D. The import statement is incorrect.

Solution

1. Step 1: Check parameter names for RandomForestClassifier

   The correct parameter for number of trees is n_estimators, not n_trees.

2. Step 2: Verify other parts of the code

   fit method exists, import is correct, and lists can work if compatible.

3. Final Answer:

   The parameter name should be n_estimators, not n_trees. -> Option A

4. Quick Check:

   Use n_estimators for tree count [OK]

Hint: Use n_estimators to set number of trees [OK]

Common Mistakes:

• Using wrong parameter names
• Thinking fit method is missing
• Assuming data must be numpy arrays

5. You want to improve your random forest classifier's performance on a dataset with many features and some noise. Which combination of hyperparameters is best to try first?

hard

A. Decrease n_estimators and remove max_depth limit to grow full trees.

B. Increase n_estimators and limit max_depth to prevent overfitting.

C. Set max_features to total features and use only 1 tree.

D. Use default parameters without changes.

Solution

1. Step 1: Understand the effect of n_estimators and max_depth

   More trees (higher n_estimators) reduce variance. Limiting max_depth controls overfitting on noisy data.

2. Step 2: Evaluate options for noisy, high-feature data

   Increasing trees and limiting depth balances bias and variance well for such data.

3. Final Answer:

   Increase n_estimators and limit max_depth to prevent overfitting. -> Option B

4. Quick Check:

   More trees + depth limit = better generalization [OK]

Hint: More trees + limit depth to handle noise [OK]

Common Mistakes:

• Using too few trees causing unstable results
• Growing full trees causing overfitting
• Ignoring hyperparameter tuning