One-hot encoding in ML Python - ML Experiment: Train & Evaluate

Experiment - One-hot encoding

Problem:You have a dataset with a categorical feature 'Color' having values like 'Red', 'Green', and 'Blue'. You want to convert this feature into a format that a machine learning model can understand.

Current Metrics:The model trained on raw categorical data without encoding achieves 60% accuracy on validation data.

Issue:The model cannot interpret categorical text data directly, leading to poor performance.

Your Task

Apply one-hot encoding to the 'Color' feature to improve model accuracy to at least 75%.

Use one-hot encoding only on the 'Color' feature.

Keep the rest of the dataset and model architecture unchanged.

Hint 1

Hint 2

Solution

ML Python

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

# Sample dataset
data = pd.DataFrame({
    'Color': ['Red', 'Green', 'Blue', 'Green', 'Red', 'Blue', 'Red', 'Green'],
    'Size': [1, 2, 3, 2, 1, 3, 1, 2],
    'Label': [0, 1, 0, 1, 0, 0, 0, 1]
})

# One-hot encode the 'Color' feature
color_encoded = pd.get_dummies(data['Color'], prefix='Color')

# Replace 'Color' column with encoded columns
data_encoded = pd.concat([data.drop('Color', axis=1), color_encoded], axis=1)

# Split features and target
X = data_encoded.drop('Label', axis=1)
y = data_encoded['Label']

# Split data
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.25, random_state=42)

# Train model
model = LogisticRegression()
model.fit(X_train, y_train)

# Predict and evaluate
y_pred = model.predict(X_val)
accuracy = accuracy_score(y_val, y_pred)

print(f"Validation Accuracy after one-hot encoding: {accuracy * 100:.2f}%")

Applied one-hot encoding to the 'Color' categorical feature using pandas get_dummies.

Replaced the original 'Color' column with the new binary columns representing each category.

Trained the same logistic regression model on the encoded data.

Results Interpretation

Before one-hot encoding, the model accuracy was 60%. After encoding, accuracy improved to 100% on validation data.

This shows the model better understands categorical data when it is converted into a numeric format it can process.

One-hot encoding transforms categorical text data into a numeric format that machine learning models can use effectively, often improving model accuracy.

Bonus Experiment

Try using label encoding instead of one-hot encoding on the 'Color' feature and compare the model accuracy.

💡 Hint

Label encoding assigns a unique number to each category but may introduce unintended order. Observe how this affects model performance.

Practice

1. What does one-hot encoding do in machine learning?

easy

A. It converts categorical labels into binary columns with 1s and 0s.

B. It normalizes numerical data to a 0-1 range.

C. It reduces the number of features by combining categories.

D. It fills missing values with the most frequent category.

Solution

1. Step 1: Understand the purpose of one-hot encoding

   One-hot encoding transforms categorical data into a format that machine learning models can use by creating separate binary columns for each category.

2. Step 2: Compare options with this definition

   Only It converts categorical labels into binary columns with 1s and 0s. describes this process correctly; others describe different preprocessing steps.

3. Final Answer:

   It converts categorical labels into binary columns with 1s and 0s. -> Option A

4. Quick Check:

   One-hot encoding = binary columns [OK]

Hint: One-hot means one column per category with 1 or 0 [OK]

Common Mistakes:

• Confusing one-hot encoding with normalization
• Thinking it reduces features instead of expanding
• Mixing it up with missing value imputation

2. Which of the following is the correct way to apply one-hot encoding using pandas in Python?

easy

A. data.encode_onehot('color')

B. data.one_hot_encode('color')

C. pd.onehot(data['color'])

D. pd.get_dummies(data['color'])

Solution

1. Step 1: Recall pandas function for one-hot encoding

   The pandas library uses the function get_dummies() to perform one-hot encoding on a column.

2. Step 2: Match the correct syntax

   Only pd.get_dummies(data['color']) uses the correct function and syntax; other options are invalid pandas methods.

3. Final Answer:

   pd.get_dummies(data['color']) -> Option D

4. Quick Check:

   pandas one-hot = get_dummies() [OK]

Hint: Use pd.get_dummies() for one-hot encoding in pandas [OK]

Common Mistakes:

• Using non-existent pandas methods
• Trying to call one-hot encoding directly on DataFrame without get_dummies
• Confusing method names

3. Given the code:

import pandas as pd
colors = ['red', 'blue', 'green', 'blue']
df = pd.DataFrame({'color': colors})
encoded = pd.get_dummies(df['color'])
print(encoded)

What is the printed output?

medium

A. A list of encoded numbers like [0,1,2,1].

B. An error because get_dummies requires a DataFrame, not a Series.

C. A DataFrame with columns 'red', 'blue', 'green' containing 1s and 0s for each row.

D. A DataFrame with a single column showing the original colors.

Solution

1. Step 1: Understand pd.get_dummies on a Series

   Applying pd.get_dummies on a Series creates a DataFrame with one column per unique category, filled with 1s and 0s indicating presence.

2. Step 2: Predict the output for given colors

   Since colors are 'red', 'blue', 'green', 'blue', the output will have columns 'blue', 'green', 'red' with 1s where the color matches and 0s otherwise.

3. Final Answer:

   A DataFrame with columns 'red', 'blue', 'green' containing 1s and 0s for each row. -> Option C

4. Quick Check:

   get_dummies output = binary columns DataFrame [OK]

Hint: get_dummies creates one column per category with 1/0 [OK]

Common Mistakes:

• Expecting numeric labels instead of binary columns
• Thinking get_dummies returns a list
• Assuming get_dummies needs a DataFrame, not Series

4. You wrote this code to one-hot encode a column but get an error:

from sklearn.preprocessing import OneHotEncoder
encoder = OneHotEncoder()
encoder.fit(['red', 'blue', 'green'])

What is the error and how to fix it?

medium

A. Error: OneHotEncoder requires numeric input; convert colors to numbers first.

B. Error: input must be 2D array; fix by reshaping input to [['red'], ['blue'], ['green']].

C. Error: OneHotEncoder is deprecated; use pd.get_dummies instead.

D. No error; code runs fine as is.

Solution

1. Step 1: Identify input shape requirement for OneHotEncoder

   sklearn's OneHotEncoder expects a 2D array (like a list of lists), not a 1D list.

2. Step 2: Fix input shape

   Reshape the input to [['red'], ['blue'], ['green']] to make it 2D and avoid the error.

3. Final Answer:

   Error: input must be 2D array; fix by reshaping input to [['red'], ['blue'], ['green']]. -> Option B

4. Quick Check:

   OneHotEncoder input = 2D array [OK]

Hint: OneHotEncoder needs 2D input, reshape 1D list to list of lists [OK]

Common Mistakes:

• Passing 1D list instead of 2D array
• Thinking OneHotEncoder only works with numbers
• Ignoring sklearn input shape requirements

5. You have a dataset with a column 'fruit' containing ['apple', 'banana', 'apple', 'orange', 'banana', 'apple']. You want to one-hot encode it but also keep track of the original order and avoid creating extra columns for unseen fruits later. Which approach is best?

hard

A. Use sklearn's OneHotEncoder with handle_unknown='ignore' and fit on training data only.

B. Use pd.get_dummies on the entire dataset including test data.

C. Manually create columns for each fruit and fill 1 or 0 by checking each row.

D. Convert fruits to numbers using label encoding before one-hot encoding.

Solution

1. Step 1: Understand the need to handle unseen categories

   When encoding training data, unseen categories in test data can cause errors unless handled properly.

2. Step 2: Choose method that fits training data and ignores unknowns

   sklearn's OneHotEncoder with handle_unknown='ignore' fits on training data and safely encodes test data without errors.

3. Step 3: Avoid pd.get_dummies on combined data to prevent data leakage

   Using pd.get_dummies on all data leaks test info into training and may create inconsistent columns.

4. Final Answer:

   Use sklearn's OneHotEncoder with handle_unknown='ignore' and fit on training data only. -> Option A

5. Quick Check:

   OneHotEncoder with ignore unknown = best practice [OK]

Hint: Fit encoder on train, ignore unknown categories in test [OK]

Common Mistakes:

• Using pd.get_dummies on combined train and test data
• Not handling unknown categories causing errors
• Label encoding before one-hot causing wrong model input