Bird
Raised Fist0

Handling Class Imbalance - Oversampling, Undersampling, Focal Loss

THEORYintermediateInterview freq: very_high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
🐛
The Silent Recall Failure
Issue noticed after the last model update two weeks ago

Fraud detection model for a financial service processing 500K transactions daily

Symptom
Model accuracy is stable at 0.95, but recall for the minority class is 0
📡
Signals
metric
Recall for the minority class in the validation set
Recall dropped from 0.75 to 0 after the last update
distribution
Class distribution in the training data
Minority class instances reduced from 10% to 1% after data preprocessing
feature_importance
Feature importance for 'transaction_amount'
Importance dropped from 0.22 to 0.05 post-update
log
Training log showing oversampling step
Oversampling step skipped due to a configuration error
metric
AUC score on validation data
AUC remained stable at 0.92
🔬
Hypotheses
The model is not handling class imbalance correctly due to a skipped oversampling step
Supporting Signals
Recall for the minority classTraining log showing oversampling step
Verify: Check the training pipeline configuration for oversampling settings
assert 'oversampling' in config and config['oversampling'] == True
Feature 'transaction_amount' is no longer predictive due to data preprocessing issues
Supporting Signals
Feature importance for 'transaction_amount'
Verify: Analyze preprocessing steps applied to 'transaction_amount'
df['transaction_amount'].describe()
🎯
Root Cause
Oversampling step was skipped due to a configuration error in the training pipeline
Why It Happened

A recent update to the training pipeline inadvertently disabled the oversampling configuration

🔧
Fix Strategy
ImmediateRollback to the previous model version
Short-termCorrect the configuration to ensure oversampling is applied
Long-termImplement automated checks to verify data preprocessing steps before training
🐍
Python Fix
Python
config['oversampling'] = True
# Re-run the training pipeline
train_model(config)
✗
Common Wrong Diagnoses
▼ Show
The model is overfitting to the majority class
Why candidates think this: High accuracy might suggest overfitting to the majority class
Why wrong: The stable AUC and zero recall indicate a preprocessing issue, not overfitting
Feature 'transaction_amount' is incorrectly scaled
Why candidates think this: Drop in feature importance suggests scaling issues
Why wrong: The main issue is the skipped oversampling, not feature scaling
📊
Monitoring to Add
  • Daily checks on class distribution in training data
  • Automated alerts for significant drops in recall
  • Validation of configuration settings before model training
📋
Process Change

Introduce a checklist for configuration verification before deployment

🏗️
Architectural Change

Develop a robust data preprocessing pipeline with built-in validation checks

💡
Key Insight
Skipping critical preprocessing steps like oversampling can silently degrade model performance on minority classes.

Practice

(1/5)
1. Given the following Python code using Word2Vec, what is the output shape of the word vector for 'king'? 
from gensim.models import Word2Vec
model = Word2Vec(sentences, vector_size=100, window=5, min_count=1, workers=4)
vector = model.wv['king']
easy
A. (100, 1)
B. (1, 100)
C. (100,)
D. (1,)

Solution

  1. Step 1: Understand Word2Vec output

    The Word2Vec model returns a 1D array of the specified vector size for each word.

  2. Step 2: Determine the shape

    The vector size is set to 100, so the output shape is (100,).

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Word2Vec outputs 1D vector of size [OK]
Hint: Word2Vec outputs 1D vector of size [OK]
Common Mistakes:
2. In the context of high-dimensional data, which of the following is a primary reason why the curse of dimensionality affects nearest neighbor algorithms?
easy
A. Distances between points become less meaningful as dimensions increase.
B. High-dimensional data increases the risk of overfitting.
C. High-dimensional data requires more computational resources.
D. High-dimensional data leads to more complex decision boundaries.

Solution

  1. Step 1: Understand the curse of dimensionality.

    In high dimensions, the distance between points becomes less informative.

  2. Step 2: Eliminate wrong options.

    High-dimensional data increases the risk of overfitting. is about overfitting, not distance. High-dimensional data requires more computational resources. is about computation, not distance. High-dimensional data leads to more complex decision boundaries. is about decision boundaries, not distance.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    High dimensions dilute distance meaning. [OK]
Hint: High dimensions dilute distance meaning. [OK]
Common Mistakes:
3. In the context of the No Free Lunch Theorem, which statement is true about model performance across different datasets?
easy
A. A model that performs well on one dataset will perform well on all datasets.
B. No single model performs best on all possible datasets.
C. The best model for a dataset can be determined without testing.
D. A complex model always outperforms a simple model.

Solution

  1. Step 1: Understand the No Free Lunch Theorem

    The theorem states that no single model is universally best across all datasets.

  2. Step 2: Eliminate wrong options

    A model that performs well on one dataset will perform well on all datasets. is incorrect because model performance varies by dataset. The best model for a dataset can be determined without testing. is incorrect because testing is necessary to determine the best model. A complex model always outperforms a simple model. is incorrect because complexity does not guarantee better performance.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    No universal best model. [OK]
Hint: No universal best model. [OK]
Common Mistakes:
4. In a real-time system where predictions need to be made quickly, which approach to linear regression is more suitable and why?
hard
A. Stochastic Gradient Descent because it updates weights more frequently
B. Least Squares because it provides the exact solution
C. Batch Gradient Descent because it processes the entire dataset at once
D. Mini-batch Gradient Descent because it balances speed and accuracy

Solution

  1. Step 1: Consider real-time constraints

    Stochastic Gradient Descent (SGD) updates weights after each data point, allowing for faster, more frequent updates suitable for real-time systems.

  2. Step 2: Eliminate wrong options

    Batch Gradient Descent (A) is too slow for real-time, Least Squares (C) is not iterative, and Mini-batch (D) is a compromise but not as fast as SGD.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    SGD updates weights after each data point. [OK]
Hint: SGD updates weights after each data point. [OK]
Common Mistakes:
5. You need to design a recurrent neural network for a real-time translation system. Which approach would best handle the vanishing gradient problem while maintaining low latency?
hard
A. Use a deep LSTM with many layers.
B. Use a shallow GRU network.
C. Use a standard RNN with gradient clipping.
D. Use a bidirectional LSTM with attention mechanism.

Solution

  1. Step 1: Consider latency and vanishing gradient.

    GRUs are simpler and faster, reducing latency while handling vanishing gradients.

  2. Step 2: Evaluate other options.

    Deep LSTM (A) increases latency, standard RNN (C) struggles with gradients, bidirectional LSTM (D) adds complexity and latency.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    GRU: fast, handles vanishing gradients. [check]
Hint: GRU: fast, handles vanishing gradients. [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard