Bird
Raised Fist0

Sampling Strategies - Random vs Stratified

THEORYintermediateInterview freq: high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
⚖️
Random Sampling vs Stratified Sampling

When dealing with imbalanced datasets, would you choose random or stratified sampling and why?

📊
Comparison
DimensionRandom SamplingStratified Sampling
Implementation ComplexityLowMedium
Class BalancePoor if imbalancedGood
Computational CostLowMedium
Sample VarianceHighLow
Minority Class RepresentationPoorGood
✅
Verdict
Default to random sampling for large, balanced datasets due to its simplicity and low computational cost. Switch to stratified sampling when dealing with small, imbalanced datasets where minority class representation is crucial.
💡
Key Insight
Choose stratified sampling to maintain class distribution in samples, especially when minority class performance is critical.
⚠️
Interview Trap
❌ Wrong: Always use stratified sampling for imbalanced datasets.
Ignoring the computational cost and complexity for large datasets where random sampling might suffice.
✅ Consider the dataset size and class distribution; use stratified sampling when minority class representation is critical and dataset size is manageable.
Random Sampling
"Simplicity and ease of implementation"
Weaknesses
  • Can lead to imbalanced subsets if the original data is imbalanced
  • May not capture minority class characteristics well
  • Potentially high variance in sample statistics
  • Not suitable for small datasets with rare events
Ideal When
  • When n > 100K and class distribution is balanced
  • When computational resources are limited
  • When quick prototyping is needed
  • When the dataset is homogeneous
Google uses random sampling for A/B testing where balanced class distribution is ensured by design.
Stratified Sampling
"Maintains class distribution in samples"
Weaknesses
  • More complex to implement than random sampling
  • Requires knowledge of class labels
  • Can be computationally expensive for large datasets
  • Not effective if class distribution is already balanced
Ideal When
  • When n < 10K and class imbalance is > 1:10
  • When model performance on minority classes is critical
  • When dataset is highly heterogeneous
  • When regulatory compliance requires balanced representation
Facebook uses stratified sampling for training models on imbalanced datasets to ensure minority class representation.
🐍
Random Sampling - Example
▼ Show
Python
import numpy as np

data = np.arange(1000)
sample_size = 100
random_sample = np.random.choice(data, sample_size, replace=False)
🐍
Stratified Sampling - Example
▼ Show
Python
from sklearn.model_selection import train_test_split
import numpy as np

X = np.arange(1000).reshape(-1, 1)
y = np.random.choice([0, 1], size=1000, p=[0.9, 0.1])
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y)
✅
Use Random Sampling When
  • •When n > 100K and class distribution is balanced
  • •When computational resources are limited
  • •When quick prototyping is needed
✅
Use Stratified Sampling When
  • •When n < 10K and class imbalance is > 1:10
  • •When model performance on minority classes is critical
  • •When dataset is highly heterogeneous
↳
Nuances
  • Random sampling can be misleading in imbalanced datasets, leading to poor model performance.
  • Stratified sampling may not be necessary if the dataset is already balanced.
  • In very large datasets, the computational overhead of stratified sampling can outweigh its benefits.
  • For small datasets, stratified sampling can significantly improve model performance by ensuring minority class representation.

Practice

(1/5)
1. 
from sklearn.linear_model import Ridge
import numpy as np

X = np.array([[1, 2], [3, 4], [5, 6]])
y = np.array([1, 2, 3])
model = Ridge(alpha=0.0)
model.fit(X, y)
print(model.coef_)
What does this code output?
easy
A. An error due to alpha being zero
B. The same coefficients as ordinary least squares
C. Zero coefficients due to zero alpha
D. Random coefficients due to no regularization

Solution

  1. Step 1: Understand the effect of alpha=0 in Ridge.

    Setting alpha to zero in Ridge regression makes it equivalent to ordinary least squares.

  2. Step 2: Eliminate wrong options.

    Alpha=0 does not cause an error. Zero coefficients would require L1 regularization. Random coefficients are incorrect as the model still fits the data.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Alpha=0 in Ridge = OLS. [OK]
Hint: Alpha=0 in Ridge = OLS. [OK]
Common Mistakes:
2. A model using Batch Normalization is showing unstable training behavior with fluctuating loss. What is the most likely cause?
medium
A. The batch size is too small, leading to unreliable batch statistics.
B. The learning rate is too high, causing instability.
C. The model architecture is too deep, causing vanishing gradients.
D. The dataset is imbalanced, affecting the normalization process.

Solution

  1. Step 1: Identify the role of batch size in Batch Normalization.

    Batch Normalization relies on accurate batch statistics, which can be unreliable with small batch sizes.

  2. Step 2: Eliminate other causes.

    The learning rate is too high, causing instability. is a common issue but not specific to Batch Norm. The model architecture is too deep, causing vanishing gradients. is unrelated to normalization. The dataset is imbalanced, affecting the normalization process. affects model performance but not directly related to Batch Norm instability.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Small batch size causes unstable Batch Norm. [OK]
Hint: Small batch size causes unstable Batch Norm. [OK]
Common Mistakes:
3. When choosing between online and offline data collection for a recommendation system, which factor is most critical for deciding to use online data collection?
medium
A. The need for real-time user feedback integration.
B. The availability of large historical datasets.
C. The requirement for high computational resources.
D. The simplicity of the data processing pipeline.

Solution

  1. Step 1: Identify the key factor for online data collection

    Online data collection is crucial when real-time feedback is needed to update recommendations dynamically.

  2. Step 2: Eliminate factors not specific to online collection

    Options B, C, and D do not specifically require online data collection; they can be addressed with offline methods.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Real-time needs = online collection. [OK]
Hint: Real-time needs = online collection. [OK]
Common Mistakes:
4. A deep network is underperforming on a complex task despite having sufficient data. What is a likely cause?
medium
A. The network is too deep, causing overfitting.
B. The batch size is too small, causing slow convergence.
C. The learning rate is too high, causing divergence.
D. The network is too shallow, missing hierarchical features.

Solution

  1. Step 1: Analyze the symptoms

    Underperformance on complex tasks often indicates a lack of capacity to capture hierarchical features, suggesting the network might be too shallow.

  2. Step 2: Eliminate wrong options

    The network is too deep, causing overfitting. is incorrect because overfitting would show high training accuracy. The learning rate is too high, causing divergence. is incorrect because divergence would cause erratic loss. The batch size is too small, causing slow convergence. is incorrect because slow convergence would not cause underperformance if training is complete.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Shallow networks miss hierarchies. [OK]
Hint: Shallow networks miss hierarchies. [OK]
Common Mistakes:
5. How would you adapt K-Fold cross-validation for a dataset with a temporal component and class imbalance?
hard
A. Use standard K-Fold and shuffle the data to ensure randomness.
B. Use Stratified K-Fold without shuffling to maintain class balance.
C. Use Leave-One-Out to maximize training data and handle imbalance.
D. Use Time Series Split and manually balance classes in each fold.

Solution

  1. Step 1: Address both temporal order and class imbalance.

    Time Series Split respects temporal order, and manual balancing addresses class imbalance.

  2. Step 2: Eliminate wrong options.

    Standard K-Fold (A) ignores temporal order, Stratified K-Fold (B) doesn't respect temporal order, Leave-One-Out (D) is computationally expensive and doesn't address temporal order.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Time series + imbalance = manual balance. [OK]
Hint: Time series + imbalance = manual balance. [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard