Threshold tuning in ML Python

Introduction

Threshold tuning helps decide the best cutoff point to say if a prediction is positive or negative. It improves how well a model makes decisions.

When you want to balance between catching all positive cases and avoiding false alarms.

When the cost of wrong decisions is different for positive and negative results.

When your model gives probabilities but you need clear yes/no answers.

When you want to improve metrics like precision, recall, or F1 score.

When you want to customize the model behavior for specific business needs.

Syntax

ML Python

for threshold in thresholds:
    predictions = (probabilities >= threshold).astype(int)
    metric_value = metric(true_labels, predictions)

Thresholds are values between 0 and 1.

Probabilities come from model outputs like logistic regression or neural networks.

Examples

Use 0.5 as the cutoff to decide positive or negative.

ML Python

threshold = 0.5
predictions = (probabilities >= threshold).astype(int)

Try multiple thresholds to find the best one.

ML Python

thresholds = [0.3, 0.5, 0.7]
for t in thresholds:
    preds = (probabilities >= t).astype(int)

Select the threshold that gives the highest metric score.

ML Python

best_threshold = thresholds[np.argmax(metric_scores)]

Sample Model

This code tests thresholds from 0.0 to 1.0 in steps of 0.1. It calculates the F1 score for each threshold and finds the best one.

ML Python

import numpy as np
from sklearn.metrics import f1_score

# True labels
true_labels = np.array([0, 1, 0, 1, 1, 0, 1, 0])

# Model predicted probabilities
probabilities = np.array([0.1, 0.4, 0.35, 0.8, 0.7, 0.2, 0.9, 0.05])

# Define thresholds to test
thresholds = np.arange(0.0, 1.01, 0.1)

best_threshold = 0.0
best_f1 = 0.0

for threshold in thresholds:
    predictions = (probabilities >= threshold).astype(int)
    score = f1_score(true_labels, predictions)
    print(f"Threshold: {threshold:.1f}, F1 Score: {score:.2f}")
    if score > best_f1:
        best_f1 = score
        best_threshold = threshold

print(f"\nBest threshold: {best_threshold:.1f} with F1 Score: {best_f1:.2f}")

Important Notes

Lower thresholds catch more positives but may increase false alarms.

Higher thresholds reduce false alarms but may miss positives.

Choose threshold based on what matters more: catching positives or avoiding false alarms.

Summary

Threshold tuning helps pick the best cutoff for yes/no decisions from probabilities.

Try different thresholds and check metrics like F1 score to find the best one.

Adjust threshold to balance between catching positives and avoiding false alarms.