Handling imbalanced text data in NLP

Introduction

Imbalanced text data means some categories have many examples, and others have very few. Handling this helps models learn better and make fair predictions.

When classifying emails as spam or not spam, but spam emails are much fewer.

When detecting rare events in customer reviews, like complaints versus normal feedback.

When sorting news articles into topics where some topics appear less often.

When building sentiment analysis models with many neutral texts but few positive or negative ones.

Syntax

NLP

from sklearn.utils import resample

# Upsample minority class
minority_upsampled = resample(minority_class_data, replace=True, n_samples=desired_count, random_state=42)

# Downsample majority class
majority_downsampled = resample(majority_class_data, replace=False, n_samples=desired_count, random_state=42)

Use resample to increase or decrease samples in classes.

Set random_state for reproducible results.

Examples

This makes the smaller class as big as the larger one by repeating samples.

NLP

from sklearn.utils import resample

# Upsample minority class to match majority
minority_upsampled = resample(minority_data, replace=True, n_samples=len(majority_data), random_state=1)

This reduces the larger class size by randomly picking fewer samples.

NLP

from sklearn.utils import resample

# Downsample majority class to match minority
majority_downsampled = resample(majority_data, replace=False, n_samples=len(minority_data), random_state=1)

SMOTE creates new synthetic samples for the minority class to balance data.

NLP

from imblearn.over_sampling import SMOTE

smote = SMOTE(random_state=42)
X_resampled, y_resampled = smote.fit_resample(X, y)

Sample Model

This example shows how to balance a small text dataset by repeating the minority class samples, then train a simple model and check its performance.

NLP

from sklearn.feature_extraction.text import CountVectorizer
from sklearn.utils import resample
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report

# Sample imbalanced text data
texts = ["good product", "bad product", "excellent", "poor quality", "nice", "terrible", "awesome", "bad", "great", "awful"]
labels = [1, 0, 1, 0, 1, 0, 1, 0, 1, 0]  # 1=positive, 0=negative

# Separate majority and minority classes
texts_pos = [t for t, l in zip(texts, labels) if l == 1]
texts_neg = [t for t, l in zip(texts, labels) if l == 0]

# Upsample minority class (positive) to match majority (negative)
texts_pos_upsampled = resample(texts_pos, replace=True, n_samples=len(texts_neg), random_state=42)
labels_pos_upsampled = [1] * len(texts_pos_upsampled)

# Combine balanced data
texts_balanced = texts_neg + texts_pos_upsampled
labels_balanced = [0] * len(texts_neg) + labels_pos_upsampled

# Vectorize text
vectorizer = CountVectorizer()
X = vectorizer.fit_transform(texts_balanced)
y = labels_balanced

# Train logistic regression
model = LogisticRegression(random_state=42)
model.fit(X, y)

# Predict on training data
predictions = model.predict(X)

# Print classification report
report = classification_report(y, predictions, zero_division=0)
print(report)

Important Notes

Upsampling can cause overfitting because it repeats data.

Downsampling may lose useful information by removing data.

Try synthetic methods like SMOTE for better balance without duplicates.

Summary

Imbalanced text data can hurt model learning and fairness.

Use upsampling, downsampling, or synthetic sampling to balance classes.

Check model performance carefully after balancing data.

Practice

1. What is the main problem caused by imbalanced text data in machine learning models?

easy

A. The model may become biased towards the majority class

B. The model will always have perfect accuracy

C. The model will ignore all classes

D. The model will run faster

Solution

1. Step 1: Understand class imbalance impact

   Imbalanced data means one class has many more examples than others, causing the model to favor that class.

2. Step 2: Recognize bias effect

   This bias leads to poor performance on minority classes, reducing fairness and accuracy for those classes.

3. Final Answer:

   The model may become biased towards the majority class -> Option A

4. Quick Check:

   Imbalanced data causes bias = D [OK]

Hint: Imbalance means bias toward bigger class [OK]

Common Mistakes:

• Thinking imbalance improves accuracy
• Assuming model ignores all classes
• Believing imbalance speeds up training

2. Which Python library function is commonly used to perform upsampling on imbalanced text data?

easy

A. numpy.dot

B. pandas.read_csv

C. sklearn.utils.resample

D. matplotlib.plot

Solution

1. Step 1: Identify upsampling tool

   Upsampling means increasing minority class samples, and sklearn.utils.resample is designed for this.

2. Step 2: Eliminate unrelated functions

   pandas.read_csv loads data, numpy.dot does matrix multiplication, matplotlib.plot draws graphs, so they don't upsample.

3. Final Answer:

   sklearn.utils.resample -> Option C

4. Quick Check:

   Upsampling uses sklearn.utils.resample = A [OK]

Hint: Upsample with sklearn.utils.resample [OK]

Common Mistakes:

• Confusing data loading with upsampling
• Using plotting or math functions for sampling
• Not knowing sklearn utilities

3. Given this Python code snippet for downsampling the majority class in text data, what will be the length of downsampled_majority?

from sklearn.utils import resample
majority = ['a'] * 1000
minority = ['b'] * 100

downsampled_majority = resample(majority, replace=False, n_samples=len(minority), random_state=42)
print(len(downsampled_majority))

medium

A. 1000

B. 42

C. 1100

D. 100

Solution

1. Step 1: Understand resample parameters

   resample is called with n_samples equal to length of minority (100), so it will pick 100 samples from majority.

2. Step 2: Check replace and output length

   replace=False means no duplicates, so output length equals n_samples, which is 100.

3. Final Answer:

   100 -> Option D

4. Quick Check:

   Downsampled length = minority size = 100 [OK]

Hint: Downsample size matches minority length [OK]

Common Mistakes:

• Assuming output length equals original majority size
• Confusing random_state with sample size
• Ignoring n_samples parameter

4. Identify the error in this code snippet that tries to balance imbalanced text data by upsampling minority class:

from sklearn.utils import resample
minority = ['text1', 'text2']
upsampled_minority = resample(minority, replace=True, n_samples=5)
print(len(upsampled_minority))

medium

A. No error; code runs correctly and prints 5

B. Missing random_state parameter causes error

C. replace=True is invalid for resample

D. n_samples must be less than original minority size

Solution

1. Step 1: Check resample parameters

   replace=True allows sampling with replacement, so n_samples can be larger than original minority size.

2. Step 2: Verify code behavior

   random_state is optional; code runs fine and prints length 5 as expected.

3. Final Answer:

   No error; code runs correctly and prints 5 -> Option A

4. Quick Check:

   Upsampling with replacement works = A [OK]

Hint: replace=True allows larger sample size [OK]

Common Mistakes:

• Thinking random_state is mandatory
• Believing n_samples must be smaller
• Confusing replace parameter usage

5. You have a text classification dataset with 90% class A and 10% class B. After upsampling class B to balance the data, which metric should you check to ensure your model performs well on both classes?

hard

A. Accuracy only

B. Precision and recall for each class

C. Training time

D. Number of epochs

Solution

1. Step 1: Understand metric importance

   Accuracy can be misleading with imbalanced data; precision and recall show performance per class.

2. Step 2: Choose metrics for balanced evaluation

   Precision and recall help check if model correctly identifies minority class without many false positives or negatives.

3. Final Answer:

   Precision and recall for each class -> Option B

4. Quick Check:

   Balanced data needs precision & recall check = C [OK]

Hint: Check precision and recall, not just accuracy [OK]

Common Mistakes:

• Relying only on accuracy
• Ignoring class-wise metrics
• Focusing on training time or epochs