Random forest in depth in ML Python - Model Metrics & Evaluation

Metrics & Evaluation - Random forest in depth

Which metric matters for Random Forest and WHY

Random forest is often used for classification and regression. For classification, accuracy, precision, recall, and F1 score are important. Accuracy shows overall correctness. Precision tells how many predicted positives are truly positive. Recall shows how many real positives were found. F1 balances precision and recall. For regression, mean squared error (MSE) or R-squared are used to measure prediction quality.

We choose metrics based on the problem. For example, if missing a positive case is costly, recall matters more. Random forest can handle imbalanced data well, but metrics guide us to tune it properly.

Confusion Matrix for Random Forest Classification

      | Predicted Positive | Predicted Negative |
      |--------------------|--------------------|
      | True Positive (TP)  | False Negative (FN) |
      | False Positive (FP) | True Negative (TN)  |

    Example:
    TP = 85, FP = 15, TN = 90, FN = 10
    Total samples = 85 + 15 + 90 + 10 = 200

    Precision = TP / (TP + FP) = 85 / (85 + 15) = 0.85
    Recall = TP / (TP + FN) = 85 / (85 + 10) = 0.8947
    F1 Score = 2 * (Precision * Recall) / (Precision + Recall) ≈ 0.871
    Accuracy = (TP + TN) / Total = (85 + 90) / 200 = 0.875
    

Precision vs Recall Tradeoff in Random Forest

Random forest can be tuned to favor precision or recall by adjusting the decision threshold or class weights.

Example 1: Spam Email Filter
High precision is important. We want to avoid marking good emails as spam (false positives). So, we tune random forest to reduce false positives, even if some spam emails slip through (lower recall).

Example 2: Disease Detection
High recall is critical. We want to catch as many sick patients as possible, even if some healthy people are flagged (false positives). So, random forest is tuned to reduce false negatives.

This tradeoff depends on the cost of errors in the real world.

Good vs Bad Metric Values for Random Forest

Good values:

• Accuracy above 85% on balanced data
• Precision and recall both above 80%, showing balanced performance
• F1 score close to precision and recall, indicating no extreme tradeoff
• For regression, low MSE and R-squared near 1

Bad values:

• High accuracy but very low recall (e.g., recall 20%) means many positives missed
• High recall but very low precision means many false alarms
• F1 score much lower than precision or recall shows imbalance
• For regression, very high MSE or negative R-squared means poor fit

Common Pitfalls in Random Forest Metrics

• Accuracy paradox: High accuracy on imbalanced data can be misleading. For example, if 95% of data is negative, predicting all negative gives 95% accuracy but zero recall for positives.
• Data leakage: If test data leaks into training, metrics look unrealistically good.
• Overfitting: Very high training accuracy but low test accuracy means the model memorizes training data and fails to generalize.
• Ignoring class imbalance: Not adjusting for imbalance can cause poor recall or precision on minority class.

Self Check

Your random forest model has 98% accuracy but only 12% recall on fraud cases. Is it good for production? Why or why not?

Answer: No, it is not good. Although accuracy is high, the model misses 88% of fraud cases (low recall). This means many frauds go undetected, which is costly. For fraud detection, recall is more important to catch as many frauds as possible.

Key Result

Random forest evaluation focuses on balanced precision and recall to ensure reliable predictions, especially on imbalanced data.

Practice

1. What is the main advantage of using a random forest over a single decision tree?

easy

A. It reduces overfitting by averaging multiple trees.

B. It always runs faster than a single tree.

C. It requires less data to train.

D. It uses only one feature for splitting.

Solution

1. Step 1: Understand decision tree limitations

   A single decision tree can easily overfit, meaning it learns noise and performs poorly on new data.

2. Step 2: How random forest improves

   Random forest builds many trees on random subsets of data and features, then averages their results to reduce overfitting.

3. Final Answer:

   It reduces overfitting by averaging multiple trees. -> Option A

4. Quick Check:

   Random forest reduces overfitting = B [OK]

Hint: Random forest averages trees to avoid overfitting [OK]

Common Mistakes:

• Thinking random forest is always faster than one tree
• Believing it uses fewer data than a single tree
• Assuming it splits on only one feature

2. Which of the following is the correct way to create a random forest classifier in Python using scikit-learn?

easy

A. from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators=100)

B. from sklearn.tree import RandomForest
model = RandomForest(100)

C. import randomforest
model = randomforest.RandomForestClassifier(100)

D. from sklearn.ensemble import RandomForest
model = RandomForest(n_trees=100)

Solution

1. Step 1: Identify correct import

   The random forest classifier is in sklearn.ensemble as RandomForestClassifier.

2. Step 2: Check constructor usage

   We create it by calling RandomForestClassifier with n_estimators=100 to set number of trees.

3. Final Answer:

   from sklearn.ensemble import RandomForestClassifier model = RandomForestClassifier(n_estimators=100) -> Option A

4. Quick Check:

   Correct import and parameter = A [OK]

Hint: Use sklearn.ensemble.RandomForestClassifier with n_estimators [OK]

Common Mistakes:

• Importing from sklearn.tree instead of sklearn.ensemble
• Using wrong class names like RandomForest
• Passing wrong parameter names like n_trees

3. Consider this Python code using scikit-learn's random forest:

from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators=3, max_depth=2, random_state=42)
X = [[0, 0], [1, 1], [0, 1], [1, 0]]
y = [0, 1, 1, 0]
model.fit(X, y)
preds = model.predict([[0, 0], [1, 1]])
print(list(preds))

What is the output?

medium

A. [0, 0]

B. [0, 1]

C. [1, 0]

D. [1, 1]

Solution

1. Step 1: Understand training data and labels

   Input points [0,0] and [1,1] have labels 0 and 1 respectively.

2. Step 2: Predict on same points with trained model

   Random forest with 3 trees and max depth 2 will learn simple splits and predict correctly on these points.

3. Final Answer:

   [0, 1] -> Option B

4. Quick Check:

   Predictions match training labels = C [OK]

Hint: Predictions on training points usually match labels [OK]

Common Mistakes:

• Confusing input order and labels
• Assuming random forest predicts opposite labels
• Ignoring max_depth effect

4. You wrote this code but get an error:

from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators='100')
model.fit(X_train, y_train)

What is the problem?

medium

A. fit method requires extra parameters.

B. RandomForestClassifier does not have n_estimators parameter.

C. n_estimators should be an integer, not a string.

D. You must import RandomForestRegressor instead.

Solution

1. Step 1: Check parameter type for n_estimators

   n_estimators expects an integer number of trees, not a string.

2. Step 2: Identify error cause

   Passing '100' as a string causes a type error during model creation or training.

3. Final Answer:

   n_estimators should be an integer, not a string. -> Option C

4. Quick Check:

   Parameter type mismatch = A [OK]

Hint: Use integer for n_estimators, not string [OK]

Common Mistakes:

• Passing numbers as strings in parameters
• Confusing classifier and regressor classes
• Thinking fit needs extra arguments

5. You want to improve your random forest model's accuracy on a complex dataset. Which combination of hyperparameters is best to try first?

hard

A. Set max_depth to 1 and keep n_estimators low

B. Decrease n_estimators and decrease max_depth

C. Increase max_features to total features and decrease n_estimators

D. Increase n_estimators and increase max_depth

Solution

1. Step 1: Understand effect of n_estimators

   More trees (higher n_estimators) usually improve accuracy by reducing variance.

2. Step 2: Understand effect of max_depth

   Increasing max_depth allows trees to learn more complex patterns, improving accuracy on complex data.

3. Final Answer:

   Increase n_estimators and increase max_depth -> Option D

4. Quick Check:

   More trees + deeper trees = better accuracy [OK]

Hint: More trees and deeper trees usually improve accuracy [OK]

Common Mistakes:

• Reducing trees and depth lowers accuracy
• Setting max_depth too low causes underfitting
• Increasing max_features too much can cause overfitting