Encoding categorical variables in Data Analysis Python - Step-by-Step Execution

Concept Flow - Encoding categorical variables

Start with categorical data

↓

Choose encoding method

↓

Label Encoding

↓

Map categories

↓

Replace original

↓

Encoded data ready

↓

Use in model

This flow shows how categorical data is transformed by choosing an encoding method, applying it, and preparing data for modeling.

Execution Sample

Data Analysis Python

import pandas as pd
from sklearn.preprocessing import LabelEncoder

data = pd.DataFrame({'color': ['red', 'blue', 'green', 'blue']})
le = LabelEncoder()
data['color_encoded'] = le.fit_transform(data['color'])
print(data)

This code converts color names into numbers using label encoding.

Execution Table

Step	Action	Input Data	Encoding Result	Output Data
1	Start with data	{'color': ['red', 'blue', 'green', 'blue']}	None	{'color': ['red', 'blue', 'green', 'blue']}
2	Initialize LabelEncoder	None	Ready to encode	None
3	Fit and transform 'color'	['red', 'blue', 'green', 'blue']	Map: {'blue':0, 'green':1, 'red':2}	[2, 0, 1, 0]
4	Add encoded column	Original + encoded	Encoded column added	{'color': ['red', 'blue', 'green', 'blue'], 'color_encoded': [2, 0, 1, 0]}
5	Print final data	DataFrame with encoded	Shows encoded numbers	{'color': ['red', 'blue', 'green', 'blue'], 'color_encoded': [2, 0, 1, 0]}
6	End	Encoding complete	Data ready for model	Same as step 5

💡 Encoding finished after adding the encoded column to the data.

Variable Tracker

Variable	Start	After Step 3	After Step 4	Final
data['color']	['red', 'blue', 'green', 'blue']	Same	Same	Same
le	Uninitialized	LabelEncoder fitted	Same	Same
data['color_encoded']	Not present	[2, 0, 1, 0]	[2, 0, 1, 0]	[2, 0, 1, 0]

Key Moments - 3 Insights

Why does 'blue' get encoded as 0 and not 1 or 2?

LabelEncoder assigns numbers based on alphabetical order. 'blue' comes before 'green' and 'red', so it gets 0 (see execution_table step 3).

Is the original 'color' column changed after encoding?

No, the original column stays the same; encoding adds a new column (see execution_table step 4).

Can we use these encoded numbers directly in all models?

Not always. Some models treat numbers as ordered values, so one-hot encoding might be better for non-ordered categories.

Visual Quiz - 3 Questions

Look at the execution_table at step 3. What number is assigned to 'green'?

A 1

B 0

C 2

D 3

💡 Hint

Check the 'Encoding Result' column at step 3 for the mapping.

At which step is the encoded column added to the data?

A Step 2

B Step 3

C Step 4

D Step 5

💡 Hint

Look for the action 'Add encoded column' in the execution_table.

If we replaced LabelEncoder with OneHotEncoder, what would change in the output data?

A The 'color_encoded' column would have numbers 0,1,2

B Multiple new columns with 0/1 values would be added

C The original 'color' column would be removed

D No change in data

💡 Hint

One-hot encoding creates separate binary columns for each category.

Concept Snapshot

Encoding categorical variables:
- Convert text categories to numbers for models
- Label Encoding: assigns integer labels alphabetically
- One-Hot Encoding: creates binary columns per category
- Use LabelEncoder for ordinal data
- Use OneHotEncoder for nominal data
- Keep original data unless replacing

Full Transcript

Encoding categorical variables means changing text labels into numbers so computers can understand them. We start with data that has categories like colors. We pick a method: label encoding or one-hot encoding. Label encoding changes each category to a number based on alphabetical order. For example, 'blue' becomes 0, 'green' 1, and 'red' 2. We add these numbers as a new column next to the original. This helps models use the data. One-hot encoding makes new columns for each category with 0 or 1 to show presence. This is better when categories have no order. The code example shows label encoding step by step, adding a new column with numbers. Remember, the original data stays the same unless you replace it. This process prepares categorical data for machine learning.

Practice

1. What is the main purpose of encoding categorical variables in data science?

easy

A. To visualize data using charts and graphs

B. To remove missing values from the dataset

C. To convert categories into numbers so algorithms can process them

D. To normalize numerical data between 0 and 1

Solution

1. Step 1: Understand the role of encoding

   Encoding changes categories like 'red', 'blue' into numbers so computers can understand them.

2. Step 2: Identify the purpose in data science

   Machine learning models require numbers, so encoding helps prepare data for these models.

3. Final Answer:

   To convert categories into numbers so algorithms can process them -> Option C

4. Quick Check:

   Encoding = convert categories to numbers [OK]

Hint: Encoding means turning words into numbers for models [OK]

Common Mistakes:

• Confusing encoding with data cleaning
• Thinking encoding normalizes data
• Assuming encoding is for visualization

2. Which of the following is the correct way to apply label encoding using scikit-learn in Python?

easy

A. from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
data['color_encoded'] = le.fit_transform(data['color'])

B. from sklearn.preprocessing import OneHotEncoder
le = OneHotEncoder()
data['color_encoded'] = le.fit_transform(data['color'])

C. import pandas as pd
data['color_encoded'] = pd.get_dummies(data['color'])

D. data['color_encoded'] = data['color'].astype('category').cat.codes

Solution

1. Step 1: Recognize label encoding syntax

   LabelEncoder from sklearn.preprocessing is used with fit_transform on a column.

2. Step 2: Match code to correct usage

   from sklearn.preprocessing import LabelEncoder le = LabelEncoder() data['color_encoded'] = le.fit_transform(data['color']) correctly imports LabelEncoder, creates an instance, and applies fit_transform to the 'color' column.

3. Final Answer:

   from sklearn.preprocessing import LabelEncoder le = LabelEncoder() data['color_encoded'] = le.fit_transform(data['color']) -> Option A

4. Quick Check:

   LabelEncoder usage = from sklearn.preprocessing import LabelEncoder le = LabelEncoder() data['color_encoded'] = le.fit_transform(data['color']) [OK]

Hint: LabelEncoder uses fit_transform on a single column [OK]

Common Mistakes:

• Using OneHotEncoder instead of LabelEncoder
• Trying to assign get_dummies output directly to one column
• Forgetting to import LabelEncoder

3. Given the following code, what will be the output of print(encoded_df)?

import pandas as pd
from sklearn.preprocessing import OneHotEncoder

data = pd.DataFrame({'fruit': ['apple', 'banana', 'apple', 'orange']})
encoder = OneHotEncoder(sparse=False)
encoded_array = encoder.fit_transform(data[['fruit']])
encoded_df = pd.DataFrame(encoded_array, columns=encoder.categories_[0])
print(encoded_df)

medium

A. An error because OneHotEncoder cannot handle string data

B. A DataFrame with columns 'apple', 'banana', 'orange' containing 1s and 0s indicating fruit presence

C. A DataFrame with a single column containing encoded numbers like 0, 1, 2

D. A DataFrame with original 'fruit' column and encoded columns combined

Solution

1. Step 1: Understand OneHotEncoder output

   OneHotEncoder creates separate columns for each category with 1 or 0 indicating presence.

2. Step 2: Check code behavior

   Encoded array has shape (4,3) for 'apple', 'banana', 'orange'. DataFrame columns are set to these categories.

3. Final Answer:

   A DataFrame with columns 'apple', 'banana', 'orange' containing 1s and 0s indicating fruit presence -> Option B

4. Quick Check:

   OneHotEncoder output = separate 0/1 columns [OK]

Hint: OneHotEncoder creates one column per category with 0/1 [OK]

Common Mistakes:

• Expecting a single column with numbers
• Thinking OneHotEncoder fails on strings
• Assuming original column stays in encoded_df

4. Identify the error in this code snippet for label encoding and how to fix it:

from sklearn.preprocessing import LabelEncoder
data = pd.DataFrame({'color': ['red', 'blue', 'green']})
le = LabelEncoder()
encoded = le.fit_transform(data['color'])
print(encoded)

medium

A. data is a dict, not a DataFrame; convert to DataFrame before encoding

B. LabelEncoder cannot encode colors; use OneHotEncoder instead

C. fit_transform should be called on le, not on data['color']

D. Missing import for pandas; add 'import pandas as pd'

Solution

1. Step 1: Check imports and dependencies

   The code uses pd.DataFrame without importing pandas, causing NameError: name 'pd' is not defined.

2. Step 2: Fix by adding the import

   Add 'import pandas as pd' at the beginning, before defining data.

3. Final Answer:

   Missing import for pandas; add 'import pandas as pd' -> Option D

4. Quick Check:

   Missing pandas import for pd.DataFrame [OK]

Hint: Always import pandas before using pd.DataFrame [OK]

Common Mistakes:

• Not importing pandas before using pd.DataFrame
• Confusing LabelEncoder with OneHotEncoder
• Ignoring data structure requirements

5. You have a dataset with a column 'size' containing categories: 'small', 'medium', 'large', and 'medium'. You want to encode it so that 'small' = 0, 'medium' = 1, and 'large' = 2, but also keep track of the original order. Which encoding method and approach should you use?

hard

A. Use pandas Categorical with ordered=True and then convert to codes

B. Use OneHotEncoder to create separate columns for each size

C. Use LabelEncoder directly without ordering

D. Manually replace categories with numbers using a dictionary without ordering

Solution

1. Step 1: Understand the need to preserve order

   Since 'small' < 'medium' < 'large', encoding must respect this order.

2. Step 2: Use pandas Categorical with ordered=True

   Creating an ordered categorical allows converting categories to codes that reflect the order.

3. Step 3: Convert ordered categorical to codes

   Codes will be 0 for 'small', 1 for 'medium', 2 for 'large' as desired.

4. Final Answer:

   Use pandas Categorical with ordered=True and then convert to codes -> Option A

5. Quick Check:

   Ordered categorical preserves order in encoding [OK]

Hint: Use ordered pandas Categorical to keep category order [OK]

Common Mistakes:

• Using LabelEncoder which ignores order
• Using OneHotEncoder which loses order info
• Replacing categories without defining order