Bird
Raised Fist0

Implement Feature Scaling - Standardization vs Min-Max vs Robust

CODINGbeginnerInterview freq: high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
⚖️
Standardization vs Min-Max Scaling

When preparing data for a machine learning model, would you choose standardization or min-max scaling? Justify your choice.

📊
Comparison
DimensionStandardizationMin-Max Scaling
Handling OutliersPoorPoor
Data Distribution AssumptionsAssumes GaussianNo Assumptions
Feature Range ControlNoYes
Algorithm CompatibilityGood for Gaussian-basedGood for bounded inputs
Ease of ImplementationModerateEasy
✅
Verdict
Default to Standardization when dealing with large datasets and algorithms assuming Gaussian distribution. Switch to Min-Max Scaling for smaller datasets or when using neural networks that benefit from bounded input ranges.
💡
Key Insight
Choose Standardization for Gaussian assumptions and Min-Max Scaling for bounded input requirements.
⚠️
Interview Trap
❌ Wrong: Always use Min-Max scaling because it normalizes data to a fixed range.
This ignores the importance of data distribution and algorithm assumptions, which can lead to suboptimal model performance.
✅ Consider the data distribution and algorithm requirements. Use Min-Max scaling for bounded inputs and Standardization for Gaussian assumptions.
Standardization
"Centers data around zero with unit variance, making it suitable for algorithms assuming Gaussian distribution."
Weaknesses
  • Sensitive to outliers, which can skew the mean and standard deviation.
  • May not be ideal for bounded feature ranges, as it doesn't restrict data to a specific interval.
  • Requires recalculation of mean and variance for new data, complicating online learning.
Ideal When
  • When n > 10,000 and data is approximately normally distributed.
  • When using algorithms like SVM or logistic regression that assume Gaussian distribution.
  • When feature values have different units and need to be comparable.
Standardization is commonly used in financial models where features like returns are assumed to be normally distributed.
Min-Max Scaling
"Scales data to a fixed range, typically [0, 1], preserving relative distances."
Weaknesses
  • Sensitive to outliers, which can compress the range of the majority of data.
  • Does not center data, which can be problematic for algorithms sensitive to mean shifts.
  • Requires knowledge of the minimum and maximum values, which may change with new data.
Ideal When
  • When n < 10,000 and feature values need to be within a specific range.
  • When using algorithms like neural networks that perform better with bounded inputs.
  • When data is not normally distributed and relative distances are more important.
Min-Max scaling is often used in image processing where pixel values need to be normalized to [0, 1] for neural networks.
🐍
Standardization - Example
▼ Show
Python
import numpy as np

def standardize(X):
    mean = np.mean(X, axis=0)
    std = np.std(X, axis=0)
    return (X - mean) / std

X = np.array([[1, 2], [3, 4], [5, 6]])
standardized_X = standardize(X)
print(standardized_X)
🐍
Min-Max Scaling - Example
▼ Show
Python
import numpy as np

def min_max_scale(X):
    min_val = np.min(X, axis=0)
    max_val = np.max(X, axis=0)
    return (X - min_val) / (max_val - min_val)

X = np.array([[1, 2], [3, 4], [5, 6]])
scaled_X = min_max_scale(X)
print(scaled_X)
✅
Use Standardization When
  • •When n > 10,000 and data is approximately normally distributed.
  • •When using algorithms that assume Gaussian distribution.
  • •When feature values have different units and need to be comparable.
✅
Use Min-Max Scaling When
  • •When n < 10,000 and feature values need to be within a specific range.
  • •When using algorithms like neural networks that perform better with bounded inputs.
  • •When data is not normally distributed and relative distances are more important.
↳
Nuances
  • Standardization can be misleading if the data has significant outliers, as it can skew the mean and standard deviation.
  • Min-Max scaling can compress the range of the majority of data if outliers are present, leading to loss of information.
  • For neural networks, Min-Max scaling often leads to faster convergence due to bounded inputs.
  • Standardization is preferred when the algorithm assumes data is centered around zero, such as in PCA.

Practice

(1/5)
1. What is the time complexity of updating the learning rate using PyTorch's StepLR scheduler for each epoch?
easy
A. O(n^2)
B. O(n)
C. O(1)
D. O(log n)

Solution

  1. Step 1: Analyze StepLR update mechanism

    StepLR updates the learning rate by multiplying it with a constant factor, which is a constant time operation.

  2. Step 2: Determine time complexity

    Since the update is a simple multiplication, the time complexity is O(1).

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    StepLR update is a constant time operation. [OK]
Hint: StepLR update is a constant time operation. [OK]
Common Mistakes:
2. What does the following Python code output when computing the ROC curve? 
from sklearn.metrics import roc_curve
import numpy as np

y_true = np.array([0, 0, 1, 1])
y_scores = np.array([0.1, 0.4, 0.35, 0.8])
fpr, tpr, thresholds = roc_curve(y_true, y_scores)
print(fpr, tpr, thresholds)
easy
A. array([0. , 0.5, 0.5, 1. ]), array([0. , 0. , 1. , 1. ]), array([1.8, 0.8, 0.4, 0.1])
B. array([0. , 0. , 0.5, 1. ]), array([0. , 0.5, 1. , 1. ]), array([0.8, 0.4, 0.35, 0.1])
C. array([0. , 0.5, 1. ]), array([0. , 0.5, 1. ]), array([0.8, 0.35, 0.1])
D. array([0. , 0.5, 1. ]), array([0. , 1. , 1. ]), array([0.8, 0.35, 0.1])

Solution

  1. Step 1: Understand ROC curve output

    The ROC curve function returns false positive rates (fpr), true positive rates (tpr), and thresholds.

  2. Step 2: Analyze the code

    Given the scores and true labels, the ROC curve is calculated correctly, resulting in fpr, tpr, and thresholds as in array([0. , 0.5, 1. ]), array([0. , 0.5, 1. ]), array([0.8, 0.35, 0.1]).

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    ROC curve outputs fpr, tpr, thresholds [OK]
Hint: ROC curve outputs fpr, tpr, thresholds [OK]
Common Mistakes:
3. Your mAP implementation is returning unusually high values. What is the most likely cause?
medium
A. Using precision instead of recall in calculations
B. Incorrect IoU threshold for positive matches
C. Not normalizing the precision values
D. Including false negatives in the precision calculation

Solution

  1. Step 1: Identify the Symptom

    High mAP values suggest too many matches are considered correct.

  2. Step 2: Analyze Each Option

    Incorrect IoU threshold for positive matches: A low IoU threshold can cause false positives to be counted as true positives, inflating mAP. Using precision instead of recall in calculations: Would affect recall, not mAP directly. Not normalizing the precision values: Would affect the scale, not the value. Including false negatives in the precision calculation: Would lower precision, not increase it.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Check IoU threshold for false positives [OK]
Hint: Check IoU threshold for false positives [OK]
Common Mistakes:
4. In a recommendation system, why might you choose NDCG over precision at k?
medium
A. NDCG is easier to compute.
B. NDCG considers the position of relevant items.
C. NDCG is less sensitive to irrelevant items.
D. NDCG is more interpretable than precision.

Solution

  1. Step 1: Understand NDCG vs Precision

    NDCG accounts for the position of relevant items, giving higher scores to relevant items at higher ranks.

  2. Step 2: Evaluate Options

    Precision at k does not consider the order of items, only their presence.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    NDCG values position. [OK]
Hint: NDCG values position. [OK]
Common Mistakes:
5. 
def calculate_f1(confusion_matrix):
    tp = confusion_matrix[0, 0]
    fp = confusion_matrix[1, 0]
    fn = confusion_matrix[0, 1]
    precision = tp / (tp + fp)
    recall = tp / (tp + fn)
    f1_score = 2 * (precision * recall) / (precision + recall)
    return f1_score

confusion_matrix = np.array([[50, 10], [5, 35]])
print(calculate_f1(confusion_matrix))
medium
A. 0.9091
B. 0.8696
C. 0.8333
D. Throws a division by zero error

Solution

  1. Step 1: Calculate precision and recall

    Precision = 0.9091, Recall = 0.8333.

  2. Step 2: Calculate F1-score

    F1 = 2 * (0.9091 * 0.8333) / (0.9091 + 0.8333) = 0.8696.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    F1 combines precision and recall. [OK]
Hint: F1 combines precision and recall. [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard