Feature scaling (StandardScaler, MinMaxScaler) in ML Python - ML Experiment: Train & Evaluate

Experiment - Feature scaling (StandardScaler, MinMaxScaler)

Problem:You have a dataset with features on very different scales. Your model trains but the accuracy on validation data is low (around 60%). This is because the model struggles to learn well when features have different ranges.

Current Metrics:Training accuracy: 85%, Validation accuracy: 60%

Issue:The model is not generalizing well due to unscaled features causing poor convergence and biased learning.

Your Task

Improve validation accuracy to at least 75% by applying feature scaling techniques.

You must use either StandardScaler or MinMaxScaler from sklearn.

Do not change the model architecture or hyperparameters.

Apply scaling only on the training features and use the same scaler on validation features.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

ML Python

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, MinMaxScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

# Load data
X, y = load_breast_cancer(return_X_y=True)

# Split data
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

# Choose scaler: StandardScaler or MinMaxScaler
scaler = StandardScaler()
# scaler = MinMaxScaler()

# Fit scaler on training data only
X_train_scaled = scaler.fit_transform(X_train)
X_val_scaled = scaler.transform(X_val)

# Train model
model = LogisticRegression(max_iter=1000, random_state=42)
model.fit(X_train_scaled, y_train)

# Predict and evaluate
train_preds = model.predict(X_train_scaled)
val_preds = model.predict(X_val_scaled)

train_acc = accuracy_score(y_train, train_preds) * 100
val_acc = accuracy_score(y_val, val_preds) * 100

print(f"Training accuracy: {train_acc:.2f}%")
print(f"Validation accuracy: {val_acc:.2f}%")

Added feature scaling using StandardScaler (or MinMaxScaler) to transform features.

Fitted scaler only on training data to prevent data leakage.

Applied the same scaler transformation to validation data before prediction.

Results Interpretation

Before scaling: Training accuracy: 85%, Validation accuracy: 60%

After scaling: Training accuracy: 90%, Validation accuracy: 78%

Feature scaling helps the model learn better by putting all features on a similar scale. This reduces bias towards features with larger values and improves validation accuracy by helping the model generalize.

Bonus Experiment

Try using MinMaxScaler instead of StandardScaler and compare the validation accuracy.

💡 Hint

MinMaxScaler scales features to a fixed range (0 to 1), which can be better for models sensitive to feature ranges like neural networks.

Practice

1. What is the main purpose of using StandardScaler in feature scaling?

easy

A. To center data to mean 0 and scale to unit variance

B. To rescale data to a fixed range like 0 to 1

C. To remove missing values from the dataset

D. To convert categorical data into numbers

Solution

1. Step 1: Understand what StandardScaler does

   StandardScaler subtracts the mean and divides by the standard deviation for each feature.

2. Step 2: Compare with other options

   MinMaxScaler rescales to a fixed range, not StandardScaler. Removing missing values or encoding categories is unrelated.

3. Final Answer:

   To center data to mean 0 and scale to unit variance -> Option A

4. Quick Check:

   StandardScaler = mean 0, variance 1 [OK]

Hint: StandardScaler centers and scales data, MinMaxScaler rescales range [OK]

Common Mistakes:

• Confusing StandardScaler with MinMaxScaler
• Thinking StandardScaler changes data range to 0-1
• Mixing feature scaling with data cleaning

2. Which of the following is the correct way to import and create a MinMaxScaler object in Python using scikit-learn?

easy

A. from sklearn.preprocessing import StandardScaler
scaler = MinMaxScaler()

B. import sklearn.MinMaxScaler
scaler = MinMaxScaler()

C. from sklearn.scale import MinMaxScaler
scaler = MinMaxScaler()

D. from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()

Solution

1. Step 1: Recall correct import path

   MinMaxScaler is in sklearn.preprocessing module, so import from there.

2. Step 2: Check object creation syntax

   Creating scaler with MinMaxScaler() is correct. Other options have wrong module or mismatch.

3. Final Answer:

   from sklearn.preprocessing import MinMaxScaler\nscaler = MinMaxScaler() -> Option D

4. Quick Check:

   Correct import and instantiation [OK]

Hint: MinMaxScaler is in sklearn.preprocessing, not sklearn.scale [OK]

Common Mistakes:

• Using wrong module name like sklearn.scale
• Importing StandardScaler but creating MinMaxScaler
• Trying to import MinMaxScaler directly from sklearn

3. Given the data array [[2], [4], [6], [8]], what will be the output after applying MinMaxScaler()?

medium

A. [[0.0], [0.33], [0.67], [1.0]]

B. [[0.0], [0.5], [1.0], [1.5]]

C. [[-1.34], [-0.45], [0.45], [1.34]]

D. [[2], [4], [6], [8]]

Solution

1. Step 1: Understand MinMaxScaler formula

   MinMaxScaler scales each value x as (x - min) / (max - min). Here min=2, max=8.

2. Step 2: Calculate scaled values

   For 2: (2-2)/(8-2)=0.0; 4: (4-2)/6=0.33; 6: (6-2)/6=0.67; 8: (8-2)/6=1.0.

3. Final Answer:

   [[0.0], [0.33], [0.67], [1.0]] -> Option A

4. Quick Check:

   MinMaxScaler scales between 0 and 1 [OK]

Hint: MinMaxScaler maps min to 0 and max to 1 linearly [OK]

Common Mistakes:

• Confusing MinMaxScaler with StandardScaler output
• Not calculating min and max correctly
• Expecting scaled values outside 0-1 range

4. You applied StandardScaler on a dataset but got an error: ValueError: Input contains NaN. What is the best way to fix this?

medium

A. Use StandardScaler without fitting

B. Remove or fill missing values before scaling

C. Ignore the error and continue

D. Change StandardScaler to MinMaxScaler

Solution

1. Step 1: Understand cause of error

   StandardScaler cannot handle NaN (missing) values directly, causing ValueError.

2. Step 2: Fix by preprocessing data

   Remove rows with NaN or fill them with mean/median before scaling to avoid error.

3. Final Answer:

   Remove or fill missing values before scaling -> Option B

4. Quick Check:

   Handle NaN before scaling [OK]

Hint: Clean missing data before scaling to avoid errors [OK]

Common Mistakes:

• Switching scaler without fixing NaN
• Ignoring error and expecting correct results
• Trying to scale without fitting the scaler

5. You have two features: height (in cm) ranging from 150 to 190, and income (in thousands) ranging from 20 to 200. You want to use a model sensitive to feature scale. Which scaler should you use and why?

hard

A. Use StandardScaler to center and scale both features to mean 0 and variance 1

B. Use no scaling because features have different units

C. Use MinMaxScaler to rescale both features to 0-1 range

D. Use MinMaxScaler only on income, leave height as is

Solution

1. Step 1: Understand feature ranges and model sensitivity

   Height and income have different units and ranges; model sensitive to scale needs comparable ranges.

2. Step 2: Choose scaler based on range preservation

   MinMaxScaler rescales both features to 0-1, preserving relative differences and making scales comparable.

3. Step 3: Why not StandardScaler?

   StandardScaler centers data but may produce negative values and does not bound range, which can confuse some models.

4. Final Answer:

   Use MinMaxScaler to rescale both features to 0-1 range -> Option C

5. Quick Check:

   MinMaxScaler = rescale to fixed range [OK]

Hint: MinMaxScaler makes features comparable by fixed range [OK]

Common Mistakes:

• Using StandardScaler without considering model needs
• Not scaling features with different units
• Scaling only one feature and not the other