Imbalanced class handling (SMOTE, class weights) in ML Python

Introduction

Sometimes, one group in data is much bigger than others. This can make a model unfair or wrong. We use special ways to fix this so the model learns well for all groups.

When you have very few examples of one type compared to others, like fraud detection where fraud cases are rare.

When your model always guesses the big group and ignores the small group, missing important cases.

When you want your model to be fair and not biased toward the bigger group.

When you want to improve accuracy for the smaller group without losing overall performance.

Syntax

ML Python

from imblearn.over_sampling import SMOTE

smote = SMOTE(sampling_strategy='minority')
X_resampled, y_resampled = smote.fit_resample(X, y)

# For class weights in sklearn models:
model = SomeClassifier(class_weight='balanced')
model.fit(X_train, y_train)

SMOTE creates new examples for the smaller group by mixing existing ones.

Class weights tell the model to pay more attention to smaller groups during training.

Examples

This creates new samples only for the smallest class.

ML Python

smote = SMOTE(sampling_strategy='minority')
X_res, y_res = smote.fit_resample(X, y)

This makes the smaller class half the size of the bigger class after resampling.

ML Python

smote = SMOTE(sampling_strategy=0.5)
X_res, y_res = smote.fit_resample(X, y)

This sets class weights automatically based on class frequencies.

ML Python

model = LogisticRegression(class_weight='balanced')
model.fit(X_train, y_train)

This manually sets class 1 to be 5 times more important than class 0.

ML Python

model = RandomForestClassifier(class_weight={0:1, 1:5})
model.fit(X_train, y_train)

Sample Model

This program creates a dataset where one class is much smaller. It trains a logistic regression model three ways: normal, with SMOTE to add samples, and with class weights to pay more attention to the small class. It prints reports showing how well each method works.

ML Python

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
from imblearn.over_sampling import SMOTE

# Create imbalanced data
X, y = make_classification(n_classes=2, class_sep=2,
                           weights=[0.9, 0.1], n_informative=3,
                           n_redundant=1, flip_y=0, n_features=5,
                           n_clusters_per_class=1, n_samples=200, random_state=42)

# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Without handling imbalance
model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
print("Without imbalance handling:")
print(classification_report(y_test, y_pred))

# Using SMOTE
smote = SMOTE(random_state=42)
X_train_smote, y_train_smote = smote.fit_resample(X_train, y_train)
model_smote = LogisticRegression(max_iter=1000)
model_smote.fit(X_train_smote, y_train_smote)
y_pred_smote = model_smote.predict(X_test)
print("With SMOTE:")
print(classification_report(y_test, y_pred_smote))

# Using class weights
model_cw = LogisticRegression(class_weight='balanced', max_iter=1000)
model_cw.fit(X_train, y_train)
y_pred_cw = model_cw.predict(X_test)
print("With class weights:")
print(classification_report(y_test, y_pred_cw))

Important Notes

SMOTE works by creating new synthetic examples, not just copying existing ones.

Class weights are easier to use but may not always improve results as much as SMOTE.

Always check model performance on real test data to see if imbalance handling helps.

Summary

Imbalanced data can cause models to ignore small groups.

SMOTE creates new samples to balance classes.

Class weights tell the model to focus more on smaller classes.

Practice

1. What is the main purpose of using SMOTE in machine learning?

easy

A. To create synthetic samples for minority classes to balance the dataset

B. To reduce the size of the majority class by removing samples

C. To increase the number of features in the dataset

D. To randomly shuffle the dataset before training

Solution

1. Step 1: Understand SMOTE's role in imbalanced data

   SMOTE stands for Synthetic Minority Over-sampling Technique and it creates new synthetic samples for the minority class.

2. Step 2: Compare options with SMOTE's function

   Only To create synthetic samples for minority classes to balance the dataset correctly describes SMOTE's purpose to balance classes by adding synthetic minority samples.

3. Final Answer:

   To create synthetic samples for minority classes to balance the dataset -> Option A

4. Quick Check:

   SMOTE = Synthetic samples for minority [OK]

Hint: SMOTE = make new minority samples to balance [OK]

Common Mistakes:

• Thinking SMOTE removes majority samples
• Confusing SMOTE with feature engineering
• Assuming SMOTE shuffles data

2. Which of the following is the correct way to set class weights in scikit-learn's LogisticRegression?

easy

A. LogisticRegression(class_weight='balanced')

B. LogisticRegression(weight_class='balanced')

C. LogisticRegression(classweights='balanced')

D. LogisticRegression(weights='balanced')

Solution

1. Step 1: Recall scikit-learn parameter for class weights

   The correct parameter name is class_weight and it accepts 'balanced' to auto-adjust weights.

2. Step 2: Match options with correct syntax

   Only LogisticRegression(class_weight='balanced') uses the exact parameter class_weight='balanced'.

3. Final Answer:

   LogisticRegression(class_weight='balanced') -> Option A

4. Quick Check:

   Parameter name is class_weight [OK]

Hint: Use class_weight='balanced' exactly in model init [OK]

Common Mistakes:

• Using wrong parameter names like weight_class
• Misspelling class_weight
• Passing weights instead of class_weight

3. Given this code snippet using SMOTE, what will be the shape of X_resampled and y_resampled?

from imblearn.over_sampling import SMOTE
X = [[1], [2], [3], [4], [5], [6]]
y = [0, 0, 0, 1, 1, 1]
smote = SMOTE(random_state=42)
X_resampled, y_resampled = smote.fit_resample(X, y)
print(len(X_resampled), len(y_resampled))

medium

A. 8 8

B. 6 6

C. 10 10

D. 12 12

Solution

1. Step 1: Count original class samples

   Class 0 has 3 samples, class 1 has 3 samples, so dataset is balanced initially.

2. Step 2: Understand SMOTE behavior on balanced data

   SMOTE will create synthetic samples to balance minority class to majority class size. Here both classes are equal, so no new samples are needed.

3. Step 3: Check actual output

   Since classes are equal, no new samples are added. So output length remains 6.

4. Final Answer:

   6 6 -> Option B

5. Quick Check:

   Balanced classes, no new samples added [OK]

Hint: SMOTE adds samples only if classes are imbalanced [OK]

Common Mistakes:

• Assuming SMOTE always doubles data
• Ignoring original class counts
• Confusing sample count with feature count

4. You wrote this code to apply class weights but the model accuracy is very low. What is the likely error?

from sklearn.linear_model import LogisticRegression
model = LogisticRegression(class_weight={'0':1, '1':10})
model.fit(X_train, y_train)

medium

A. LogisticRegression does not support class weights

B. class_weight parameter does not accept dictionaries

C. Class weights keys should be integers, not strings

D. class_weight values must sum to 1

Solution

1. Step 1: Check class_weight dictionary keys

   Class labels in class_weight must match label types in y_train. Usually labels are integers 0 and 1, not strings '0' and '1'.

2. Step 2: Understand impact of wrong keys

   If keys are strings but labels are integers, weights won't apply correctly, causing poor model performance.

3. Final Answer:

   Class weights keys should be integers, not strings -> Option C

4. Quick Check:

   Keys must match label types [OK]

Hint: Match class_weight keys to label data types exactly [OK]

Common Mistakes:

• Using string keys instead of integer keys
• Thinking class_weight can't be a dict
• Believing weights must sum to 1

5. You have a dataset with 95% class 0 and 5% class 1. You want to train a model that handles this imbalance. Which approach is best to improve minority class recall?

hard

A. Train the model without any imbalance handling

B. Only use SMOTE without changing class weights

C. Only set class_weight='balanced' without oversampling

D. Use SMOTE to create synthetic minority samples and set class_weight='balanced' in the model

Solution

1. Step 1: Understand dataset imbalance

   With 95% vs 5%, the minority class is very small and model may ignore it.

2. Step 2: Combine SMOTE and class weights

   SMOTE creates synthetic minority samples to balance data, while class_weight='balanced' tells model to focus more on minority class during training.

3. Step 3: Why combining is best

   Using both together improves minority recall better than using either alone or ignoring imbalance.

4. Final Answer:

   Use SMOTE to create synthetic minority samples and set class_weight='balanced' in the model -> Option D

5. Quick Check:

   Combine oversampling + class weights for best minority recall [OK]

Hint: Combine SMOTE and class_weight='balanced' for best results [OK]

Common Mistakes:

• Using only one method and expecting best recall
• Ignoring imbalance completely
• Assuming oversampling alone fixes all issues