Random forest classifier in ML Python - Model Metrics & Evaluation

Metrics & Evaluation - Random forest classifier

Which metric matters for Random Forest Classifier and WHY

A random forest classifier is used to predict categories, like whether an email is spam or not. The key metrics to check are Accuracy, Precision, Recall, and F1-score.

Accuracy tells us how many predictions were correct overall. But if classes are uneven, accuracy can be misleading.

Precision shows how many predicted positives were actually positive. This matters when false alarms are costly.

Recall shows how many actual positives were found. This matters when missing a positive is bad.

F1-score balances precision and recall, giving a single score to compare models.

For random forests, these metrics help us understand how well the model separates classes and handles errors.

Confusion Matrix Example

Imagine a test with 100 samples. The confusion matrix might look like this:

      | Predicted Positive | Predicted Negative |
      |--------------------|--------------------|
      | True Positive (TP): 40 | False Positive (FP): 5 |
      | False Negative (FN): 10 | True Negative (TN): 45 |
    

Totals: TP + FP + TN + FN = 40 + 5 + 45 + 10 = 100 samples.

From this, we calculate:

• Precision = 40 / (40 + 5) = 0.89
• Recall = 40 / (40 + 10) = 0.80
• F1-score = 2 * (0.89 * 0.80) / (0.89 + 0.80) ≈ 0.84
• Accuracy = (40 + 45) / 100 = 0.85

Precision vs Recall Tradeoff with Examples

Random forests can be tuned to favor precision or recall depending on the problem.

High Precision Example: In email spam detection, high precision means fewer good emails are wrongly marked as spam. This avoids losing important emails.

High Recall Example: In medical diagnosis, high recall means catching most sick patients, even if some healthy people are flagged wrongly. Missing a sick patient is worse.

Choosing between precision and recall depends on what mistakes cost more in your situation.

What Good vs Bad Metrics Look Like for Random Forest

Good Metrics:

• Accuracy above 80% on balanced data
• Precision and recall both above 75%
• F1-score close to precision and recall, showing balance

Bad Metrics:

• High accuracy but very low recall (missing many positives)
• High recall but very low precision (many false alarms)
• F1-score much lower than precision or recall, showing imbalance

Common Pitfalls in Metrics for Random Forest

• Accuracy Paradox: High accuracy can hide poor performance if classes are imbalanced.
• Data Leakage: If test data leaks into training, metrics look unrealistically good.
• Overfitting: Very high training accuracy but low test accuracy means the model memorizes data, not generalizes.
• Ignoring Class Imbalance: Not checking precision and recall can mislead about model usefulness.

Self Check Question

Your random forest model has 98% accuracy but only 12% recall on the positive class (e.g., fraud). Is it good for production?

Answer: No, it is not good. The model misses 88% of actual positive cases, which is dangerous if catching positives is important. High accuracy is misleading here because most data is negative. You should improve recall before using this model.

Key Result

Random forest classifier performance is best judged by balanced precision, recall, and F1-score rather than accuracy alone.

Practice

1. What is the main idea behind a Random Forest Classifier?

easy

A. It clusters data without labels.

B. It uses only one decision tree for prediction.

C. It is a type of neural network.

D. It combines many decision trees to make better predictions.

Solution

1. Step 1: Understand what a random forest is

   A random forest is a group of decision trees working together.

2. Step 2: Identify the main purpose

   Combining many trees helps improve prediction accuracy and reduce errors.

3. Final Answer:

   It combines many decision trees to make better predictions. -> Option D

4. Quick Check:

   Random forest = many trees combined [OK]

Hint: Random forest means many trees working together [OK]

Common Mistakes:

• Thinking it uses only one tree
• Confusing it with neural networks
• Believing it does clustering without labels

2. Which of the following is the correct way to import and create a random forest classifier in Python using scikit-learn?

easy

A. from sklearn.tree import RandomForest
model = RandomForest(100)

B. import randomforest
model = randomforest.RandomForest(100)

C. from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators=100)

D. from sklearn.ensemble import RandomForest
model = RandomForestClassifier()

Solution

1. Step 1: Check the correct import

   The random forest classifier is in sklearn.ensemble as RandomForestClassifier.

2. Step 2: Verify the model creation syntax

   Use RandomForestClassifier(n_estimators=100) to create with 100 trees.

3. Final Answer:

   from sklearn.ensemble import RandomForestClassifier model = RandomForestClassifier(n_estimators=100) -> Option C

4. Quick Check:

   Correct import and model creation [OK]

Hint: Use sklearn.ensemble and RandomForestClassifier [OK]

Common Mistakes:

• Importing from sklearn.tree instead of ensemble
• Using wrong class names
• Missing n_estimators parameter

3. What will be the output of the following code snippet?

from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, random_state=42)
model = RandomForestClassifier(n_estimators=10, random_state=42)
model.fit(X_train, y_train)
predictions = model.predict(X_test)
print(list(predictions[:5]))

medium

A. [2, 2, 1, 0, 0]

B. [0, 1, 2, 0, 1]

C. [1, 0, 2, 1, 1]

D. [1, 1, 1, 1, 1]

Solution

1. Step 1: Understand the dataset and split

   The iris dataset has 3 classes (0,1,2). The split is random_state=42 for reproducibility.

2. Step 2: Fit and predict with random forest

   Using 10 trees and random_state=42, predictions on test data are consistent.

3. Final Answer:

   [2, 2, 1, 0, 0] -> Option A

4. Quick Check:

   Predictions match expected output with random_state=42 [OK]

Hint: Set random_state for reproducible predictions [OK]

Common Mistakes:

• Ignoring random_state causing different outputs
• Confusing class labels order
• Assuming all predictions are the same

4. Identify the error in the following code snippet that tries to train a random forest classifier:

from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_trees=100)
model.fit(X_train, y_train)

medium

A. The parameter name should be n_estimators, not n_trees.

B. RandomForestClassifier does not have a fit method.

C. X_train and y_train must be numpy arrays, not lists.

D. The import statement is incorrect.

Solution

1. Step 1: Check parameter names for RandomForestClassifier

   The correct parameter for number of trees is n_estimators, not n_trees.

2. Step 2: Verify other parts of the code

   fit method exists, import is correct, and lists can work if compatible.

3. Final Answer:

   The parameter name should be n_estimators, not n_trees. -> Option A

4. Quick Check:

   Use n_estimators for tree count [OK]

Hint: Use n_estimators to set number of trees [OK]

Common Mistakes:

• Using wrong parameter names
• Thinking fit method is missing
• Assuming data must be numpy arrays

5. You want to improve your random forest classifier's performance on a dataset with many features and some noise. Which combination of hyperparameters is best to try first?

hard

A. Decrease n_estimators and remove max_depth limit to grow full trees.

B. Increase n_estimators and limit max_depth to prevent overfitting.

C. Set max_features to total features and use only 1 tree.

D. Use default parameters without changes.

Solution

1. Step 1: Understand the effect of n_estimators and max_depth

   More trees (higher n_estimators) reduce variance. Limiting max_depth controls overfitting on noisy data.

2. Step 2: Evaluate options for noisy, high-feature data

   Increasing trees and limiting depth balances bias and variance well for such data.

3. Final Answer:

   Increase n_estimators and limit max_depth to prevent overfitting. -> Option B

4. Quick Check:

   More trees + depth limit = better generalization [OK]

Hint: More trees + limit depth to handle noise [OK]

Common Mistakes:

• Using too few trees causing unstable results
• Growing full trees causing overfitting
• Ignoring hyperparameter tuning