Bird
Raised Fist0

Feature Scaling - Normalization vs Standardization, When Each Works

THEORYbeginnerInterview freq: high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
▶
Execution Trace
Step 1/4
Normalization

Feature values are scaled to a range between 0 and 1.

🎯 Normalization is useful when you need a bounded range.
from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
scaled_data = scaler.fit_transform(data)
Standardization

Feature values are transformed to have a mean of 0 and a standard deviation of 1.

🎯 Standardization is preferred when data has varying scales.
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaled_data = scaler.fit_transform(data)
Outlier Sensitivity

Normalization is affected by outliers, causing skewed scaling.

🎯 Consider the presence of outliers when choosing scaling methods.
# Example with outliers
data_with_outliers = [[1], [2], [1000]]
scaler = MinMaxScaler()
scaled_data = scaler.fit_transform(data_with_outliers)
Robustness to Outliers

Standardization is less affected by outliers due to centering.

🎯 Standardization is often better for datasets with outliers.
# Standardization with outliers
scaler = StandardScaler()
scaled_data = scaler.fit_transform(data_with_outliers)
📊
Diagram
Diagram
┌───────────────────────────────┐
│       Feature Scaling        │
├────────────┬──────────────────┤
│ Normalization │ Standardization │
├────────────┼──────────────────┤
│ Min-Max    │ Z-score          │
│ Scaling    │ Scaling          │
├────────────┼──────────────────┤
│ [0, 1]     │ Mean = 0         │
│ Range      │ Std Dev = 1      │
├────────────┼──────────────────┤
│ Sensitive  │ Robust to        │
│ to Outliers│ Outliers         │
└────────────┴──────────────────┘
Labels
  • Normalization: Scales data to [0, 1] range
  • Standardization: Scales data to mean 0, std dev 1
  • Min-Max Scaling: Used in Normalization
  • Z-score Scaling: Used in Standardization
  • Outlier Sensitivity: Normalization is sensitive, Standardization is robust
🔑
Key Moments
▲
Q: Why is normalization sensitive to outliers?
A: Normalization scales data to a fixed range, so extreme values can disproportionately affect the scaling of other data points.
Q: When is standardization preferred over normalization?
A: Standardization is preferred when the data has varying scales or when the presence of outliers might skew the results.
Q: How does standardization handle outliers differently than normalization?
A: Standardization centers the data around the mean, reducing the impact of outliers by focusing on the distribution's spread.
Trace Summary

The diagram explains feature scaling techniques: Normalization and Standardization. Normalization scales data to a [0, 1] range, useful for bounded data but sensitive to outliers. Standardization scales data to have a mean of 0 and a standard deviation of 1, making it robust to outliers and suitable for data with varying scales. Understanding when to use each technique is crucial for preparing data for machine learning models, especially in the presence of outliers.

Primary Focus

✓ Feature Scaling Techniques

Practice

(1/5)
1. Given the following Python code snippet, what will be the output if the input data has undergone covariate drift?
import numpy as np
from sklearn.linear_model import LinearRegression

X_train = np.array([[1, 2], [2, 3], [3, 4]])
y_train = np.array([3, 5, 7])
model = LinearRegression().fit(X_train, y_train)

X_test = np.array([[4, 5], [5, 6]])
predictions = model.predict(X_test)
print(predictions)
easy
A. The predictions will be accurate.
B. The model will throw an error.
C. The predictions will be unaffected by covariate drift.
D. The predictions will be biased due to covariate drift.

Solution

  1. Step 1: Understand the impact of covariate drift.

    Covariate drift affects the input distribution, leading to biased predictions.

  2. Step 2: Eliminate wrong options.

    The model won't throw an error (The model will throw an error.) and predictions won't be accurate (The predictions will be accurate.) or unaffected (The predictions will be unaffected by covariate drift.) due to drift.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Covariate drift biases predictions [OK]
Hint: Covariate drift biases predictions [OK]
Common Mistakes:
2. In the context of why machine learning exists, which of the following problems is most suitable for a machine learning approach rather than a rule-based system?
easy
A. Predicting the next word in a sentence
B. Calculating the sum of a list of numbers
C. Sorting a list of integers
D. Checking if a number is prime

Solution

  1. Step 1: Identify the nature of the problem.

    Predicting the next word in a sentence involves understanding context and patterns, which are complex and not easily defined by rules.

  2. Step 2: Eliminate rule-based problems.

    Options A, C, and D are deterministic and can be solved with explicit algorithms.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    ML for patterns, rules for logic. [OK]
Hint: ML excels at pattern recognition, not deterministic logic. [OK]
Common Mistakes:
3. A model performs well on the validation set but poorly on the test set. What is the most likely cause?
medium
A. The test set is not representative of the validation set.
B. The validation set is too small.
C. The model is underfitting the data.
D. The model has too many parameters.

Solution

  1. Step 1: Identify the discrepancy

    Good validation performance but poor test performance suggests a mismatch between sets.

  2. Step 2: Eliminate less likely causes

    The model is underfitting the data. is incorrect as underfitting would affect both sets. The validation set is too small. is less likely unless the validation set is extremely small. The model has too many parameters. would likely affect both sets similarly.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Mismatch between validation and test sets. [OK]
Hint: Mismatch between validation and test sets. [OK]
Common Mistakes:
4. You are tasked with designing a model for a real-time translation app. Which architecture would you choose and why?
hard
A. Use a hybrid RNN-CNN model to leverage both sequence processing and speed.
B. Use a standard RNN for its simplicity and ease of implementation.
C. Use a CNN-based model for its speed in processing sequences.
D. Use a transformer model for its ability to handle long-range dependencies efficiently.

Solution

  1. Step 1: Consider the requirements of real-time translation.

    Real-time translation requires handling long-range dependencies efficiently.

  2. Step 2: Evaluate each architecture.

    Use a standard RNN for its simplicity and ease of implementation. is incorrect as RNNs are slow for long sequences. Use a CNN-based model for its speed in processing sequences. is incorrect because CNNs struggle with sequence dependencies. Use a hybrid RNN-CNN model to leverage both sequence processing and speed. is incorrect as hybrid models add complexity without solving dependency issues.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Transformers excel in long-range dependencies. [OK]
Hint: Transformers excel in long-range dependencies. [OK]
Common Mistakes:
5. How would you adapt K-Fold cross-validation for a dataset with a temporal component and class imbalance?
hard
A. Use standard K-Fold and shuffle the data to ensure randomness.
B. Use Stratified K-Fold without shuffling to maintain class balance.
C. Use Leave-One-Out to maximize training data and handle imbalance.
D. Use Time Series Split and manually balance classes in each fold.

Solution

  1. Step 1: Address both temporal order and class imbalance.

    Time Series Split respects temporal order, and manual balancing addresses class imbalance.

  2. Step 2: Eliminate wrong options.

    Standard K-Fold (A) ignores temporal order, Stratified K-Fold (B) doesn't respect temporal order, Leave-One-Out (D) is computationally expensive and doesn't address temporal order.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Time series + imbalance = manual balance. [OK]
Hint: Time series + imbalance = manual balance. [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard