Bird
Raised Fist0

Sampling Strategies - Random, Stratified, Reservoir, Weighted

THEORYintermediateInterview freq: high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
🤖
Sampling Strategies - Random, Stratified, Reservoir, Weighted
Choosing the right sampling strategy for your data needs.
theoryintermediateFreq: highData & Features
Prerequisites:Data Splits
💡
Why It Exists

Sampling strategies were developed to address the problem of biased or inefficient data selection, which can lead to skewed model training and evaluation. For example, in a dataset with imbalanced classes, random sampling might overlook minority classes, leading to poor model performance on those classes.

📖
What It Is

Sampling strategies are methods used to select a subset of data from a larger dataset. Think of it like choosing a representative group of people from a crowd to understand the crowd's overall behavior.

∑
Math Intuition
Sampling strategies aim to create a subset of data that maintains the statistical properties of the full dataset. For example, stratified sampling ensures that the proportion of each class in the sample matches the proportion in the full dataset, which helps in maintaining class balance.
⚙️
Core Mechanism

Each strategy uses a different method to select samples, aiming to achieve specific goals like fairness, efficiency, or representativeness.

📋
Step by Step
  1. Random Sampling: Selects samples purely by chance, ensuring each sample has an equal probability of being chosen.
  2. Stratified Sampling: Divides the dataset into strata (groups) and samples from each stratum proportionally.
  3. Reservoir Sampling: Maintains a fixed-size sample from a stream of data, updating the sample as new data arrives.
  4. Weighted Sampling: Assigns weights to samples and selects them based on these weights, favoring some samples over others.
🎯
Intuition
Imagine picking marbles from a jar. Random sampling is like picking any marble without looking. Stratified is like ensuring you pick a certain number of each color. Reservoir is like keeping a fixed number of marbles as more are added. Weighted is like picking marbles with different probabilities based on their size.
🐍
Python Example
Python
import numpy as np

data = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
weights = np.array([0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1])

# Random Sampling
random_sample = np.random.choice(data, size=5, replace=False)

# Stratified Sampling (assuming 2 strata)
strata1 = data[:5]
strata2 = data[5:]
stratified_sample = np.concatenate([np.random.choice(strata1, size=2, replace=False),
                                    np.random.choice(strata2, size=3, replace=False)])

# Weighted Sampling
weighted_sample = np.random.choice(data, size=5, replace=True, p=weights)

print("Random Sample:", random_sample)  # Example output: [3 7 2 9 1]
print("Stratified Sample:", stratified_sample)  # Example output: [1 3 6 8 10]
print("Weighted Sample:", weighted_sample)  # Example output: [2 2 5 9 1]
✅
When It Works
  • Random Sampling: Works well when the dataset is large and balanced, ensuring unbiased selection.
  • Stratified Sampling: Ideal for imbalanced datasets where maintaining class proportions is crucial.
  • Reservoir Sampling: Useful for streaming data where the dataset size is unknown or too large to fit in memory.
  • Weighted Sampling: Effective when certain samples are more important or informative than others.
⚠️
When It Breaks
  • Random Sampling: Fails with highly imbalanced datasets, as minority classes may be underrepresented.
  • Stratified Sampling: Breaks when strata are not well-defined or when there are too many strata with few samples.
  • Reservoir Sampling: Ineffective if the stream has a changing distribution over time, leading to outdated samples.
  • Weighted Sampling: Can be biased if weights are not accurately reflecting the importance of samples.
⇄
vs Oversampling
Use This When
Use stratified sampling when you need to maintain class proportions in a balanced way.
Use Alternative When
Use oversampling when you need to increase the representation of minority classes beyond their natural proportion.
🎯
Interview Angles
  • Interviewer asks: 'How would you handle a dataset with severe class imbalance?' → testing your understanding of stratified sampling and its benefits.
  • Interviewer asks: 'Explain how you would sample from a continuous data stream.' → assessing your knowledge of reservoir sampling and its application.
  • Interviewer asks: 'When would you choose weighted sampling over random sampling?' → evaluating your ability to identify scenarios where sample importance varies.
⚠️
Common Mistakes
Believing random sampling is always unbiased, leading to poor performance on imbalanced datasets.
Assuming stratified sampling is unnecessary for balanced datasets, missing out on potential performance improvements.
Thinking reservoir sampling is only for small datasets, overlooking its utility in streaming contexts.
Misapplying weighted sampling without proper weight calculation, resulting in biased samples.
📌
Summary
  • Sampling strategies are crucial for creating representative subsets of data.
  • Each strategy has specific use cases, such as stratified sampling for imbalanced datasets.
  • Understanding when and how each strategy breaks is key to effective data handling.

Practice

(1/5)
1. 
import torch
x = torch.tensor([-1.0, 0.0, 1.0])
output = torch.nn.functional.relu(x)
print(output)
What is the output of this code?
easy
A. tensor([0.0, 0.0, 1.0])
B. tensor([-1.0, 0.0, 1.0])
C. tensor([0.0, 0.0, 0.0])
D. tensor([-1.0, 0.0, 0.0])

Solution

  1. Step 1: Understand ReLU behavior.

    ReLU outputs zero for negative inputs and the input itself for non-negative inputs.

  2. Step 2: Apply ReLU to each element.

    For -1.0, output is 0.0; for 0.0, output is 0.0; for 1.0, output is 1.0.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    ReLU zeros negatives, keeps positives. [OK]
Hint: ReLU zeros negatives, keeps positives. [OK]
Common Mistakes:
2. A Naive Bayes model is underperforming on a dataset. What is the most likely cause?
medium
A. The features are highly correlated.
B. The features are not normalized.
C. The model has too many parameters.
D. The dataset is too small.

Solution

  1. Step 1: Identify Potential Causes

    Naive Bayes assumes feature independence, so correlation can degrade performance.

  2. Step 2: Eliminate Other Causes

    The model has too many parameters. is incorrect because Naive Bayes is not parameter-heavy. The features are not normalized. is incorrect because normalization is not critical for Naive Bayes. The dataset is too small. is incorrect because Naive Bayes can work well with small datasets.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Correlated features degrade Naive Bayes. [[check]]
Hint: Correlated features degrade Naive Bayes. [[check]]
Common Mistakes:
3. When evaluating a model on an imbalanced dataset, why might PR-AUC be preferred over ROC-AUC?
medium
A. PR-AUC is less computationally expensive.
B. PR-AUC is more informative about the performance on the minority class.
C. PR-AUC is always higher than ROC-AUC.
D. PR-AUC is unaffected by class imbalance.

Solution

  1. Step 1: Understand metric focus

    PR-AUC focuses on precision and recall, which are crucial for minority class performance.

  2. Step 2: Eliminate wrong options

    PR-AUC is not less computationally expensive (A). It is not always higher than ROC-AUC (C). It is affected by class imbalance (D).

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    PR-AUC focuses on minority class performance. [OK]
Hint: PR-AUC focuses on minority class performance. [OK]
Common Mistakes:
4. A reinforcement learning model is not improving its performance despite extensive training. What is the most likely cause?
medium
A. The model has too many parameters.
B. The model is overfitting to the training data.
C. The training data is not labeled.
D. The reward function is not well-defined.

Solution

  1. Step 1: Identify the issue.

    In reinforcement learning, a poorly defined reward function can lead to suboptimal learning.

  2. Step 2: Eliminate wrong options.

    The model has too many parameters. is incorrect because parameter count affects complexity, not learning direction. The training data is not labeled. is irrelevant as RL doesn't require labeled data. The model is overfitting to the training data. is less likely as overfitting would show initial improvement.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Poor reward function -> Poor learning [OK]
Hint: Poor reward function -> Poor learning [OK]
Common Mistakes:
5. How would you adapt a Random Forest model to improve its robustness to adversarial attacks?
hard
A. Use adversarial training with perturbed data
B. Increase the number of trees significantly
C. Reduce the depth of each tree
D. Increase the max_features parameter

Solution

  1. Step 1: Understand adversarial robustness

    Adversarial training involves training the model with perturbed data to improve robustness.

  2. Step 2: Evaluate adaptation strategies

    Use adversarial training with perturbed data is correct because adversarial training directly addresses robustness. Increase the number of trees significantly is incorrect as more trees don't inherently improve adversarial robustness. Reduce the depth of each tree is incorrect because reducing depth may not address adversarial examples. Increase the max_features parameter is incorrect as increasing max_features doesn't target adversarial robustness.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Adversarial training improves robustness. [OK]
Hint: Adversarial training improves robustness. [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard