Encoding categorical variables in Data Analysis Python - Mini Project: Build & Apply

Encoding categorical variables

📖 Scenario: You work in a small online store. You have a list of products with their categories. You want to prepare this data for a computer program that only understands numbers, not words.

🎯 Goal: You will convert the product categories from words into numbers using encoding. This helps computers understand and analyze the data better.

📋 What You'll Learn

Create a dictionary with product names as keys and their categories as values

Create a list of unique categories

Create a dictionary that assigns a unique number to each category

Create a new dictionary with products and their encoded category numbers

Print the new encoded dictionary

💡 Why This Matters

🌍 Real World

Stores and businesses often have product categories as words. Encoding them into numbers helps in building recommendation systems and sales analysis.

💼 Career

Data scientists and analysts frequently encode categorical data to prepare datasets for machine learning models.

1

Create the product data dictionary

Create a dictionary called products with these exact entries: 'T-shirt': 'Clothing', 'Jeans': 'Clothing', 'Coffee Mug': 'Kitchen', 'Notebook': 'Stationery', 'Pen': 'Stationery'.

Data Analysis Python

# Create the products dictionary with product names and categories
# Your code here

Hint

Use curly braces {} to create a dictionary. Each entry has a product name as key and category as value.

2

Create a list of unique categories

Create a list called categories that contains the unique category names from the products dictionary values.

Data Analysis Python

products = {
    'T-shirt': 'Clothing',
    'Jeans': 'Clothing',
    'Coffee Mug': 'Kitchen',
    'Notebook': 'Stationery',
    'Pen': 'Stationery'
}
# Create a list called categories with unique category names
# Your code here

Hint

Use products.values() to get all categories, then set() to get unique ones, and convert back to list.

3

Create the encoding dictionary

Create a dictionary called category_encoding that assigns a unique number to each category in the categories list. Use a for loop with variables index and category and enumerate(categories).

Data Analysis Python

products = {
    'T-shirt': 'Clothing',
    'Jeans': 'Clothing',
    'Coffee Mug': 'Kitchen',
    'Notebook': 'Stationery',
    'Pen': 'Stationery'
}
categories = list(set(products.values()))
# Create category_encoding dictionary assigning numbers to categories
# Your code here

Hint

Use enumerate() to get index and category, then assign index as value in category_encoding.

4

Create and print the encoded product categories

Create a new dictionary called encoded_products where each product name from products is a key and its value is the encoded number from category_encoding. Use a for loop with variables product and category iterating over products.items(). Then print encoded_products.

Data Analysis Python

products = {
    'T-shirt': 'Clothing',
    'Jeans': 'Clothing',
    'Coffee Mug': 'Kitchen',
    'Notebook': 'Stationery',
    'Pen': 'Stationery'
}
categories = list(set(products.values()))
category_encoding = {}
for index, category in enumerate(categories):
    category_encoding[category] = index
# Create encoded_products dictionary with product names and encoded category numbers
# Print encoded_products
# Your code here

Hint

Use a for loop over products.items() to build encoded_products. Then print it.

Practice

1. What is the main purpose of encoding categorical variables in data science?

easy

A. To visualize data using charts and graphs

B. To remove missing values from the dataset

C. To convert categories into numbers so algorithms can process them

D. To normalize numerical data between 0 and 1

Solution

1. Step 1: Understand the role of encoding

   Encoding changes categories like 'red', 'blue' into numbers so computers can understand them.

2. Step 2: Identify the purpose in data science

   Machine learning models require numbers, so encoding helps prepare data for these models.

3. Final Answer:

   To convert categories into numbers so algorithms can process them -> Option C

4. Quick Check:

   Encoding = convert categories to numbers [OK]

Hint: Encoding means turning words into numbers for models [OK]

Common Mistakes:

• Confusing encoding with data cleaning
• Thinking encoding normalizes data
• Assuming encoding is for visualization

2. Which of the following is the correct way to apply label encoding using scikit-learn in Python?

easy

A. from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
data['color_encoded'] = le.fit_transform(data['color'])

B. from sklearn.preprocessing import OneHotEncoder
le = OneHotEncoder()
data['color_encoded'] = le.fit_transform(data['color'])

C. import pandas as pd
data['color_encoded'] = pd.get_dummies(data['color'])

D. data['color_encoded'] = data['color'].astype('category').cat.codes

Solution

1. Step 1: Recognize label encoding syntax

   LabelEncoder from sklearn.preprocessing is used with fit_transform on a column.

2. Step 2: Match code to correct usage

   from sklearn.preprocessing import LabelEncoder le = LabelEncoder() data['color_encoded'] = le.fit_transform(data['color']) correctly imports LabelEncoder, creates an instance, and applies fit_transform to the 'color' column.

3. Final Answer:

   from sklearn.preprocessing import LabelEncoder le = LabelEncoder() data['color_encoded'] = le.fit_transform(data['color']) -> Option A

4. Quick Check:

   LabelEncoder usage = from sklearn.preprocessing import LabelEncoder le = LabelEncoder() data['color_encoded'] = le.fit_transform(data['color']) [OK]

Hint: LabelEncoder uses fit_transform on a single column [OK]

Common Mistakes:

• Using OneHotEncoder instead of LabelEncoder
• Trying to assign get_dummies output directly to one column
• Forgetting to import LabelEncoder

3. Given the following code, what will be the output of print(encoded_df)?

import pandas as pd
from sklearn.preprocessing import OneHotEncoder

data = pd.DataFrame({'fruit': ['apple', 'banana', 'apple', 'orange']})
encoder = OneHotEncoder(sparse=False)
encoded_array = encoder.fit_transform(data[['fruit']])
encoded_df = pd.DataFrame(encoded_array, columns=encoder.categories_[0])
print(encoded_df)

medium

A. An error because OneHotEncoder cannot handle string data

B. A DataFrame with columns 'apple', 'banana', 'orange' containing 1s and 0s indicating fruit presence

C. A DataFrame with a single column containing encoded numbers like 0, 1, 2

D. A DataFrame with original 'fruit' column and encoded columns combined

Solution

1. Step 1: Understand OneHotEncoder output

   OneHotEncoder creates separate columns for each category with 1 or 0 indicating presence.

2. Step 2: Check code behavior

   Encoded array has shape (4,3) for 'apple', 'banana', 'orange'. DataFrame columns are set to these categories.

3. Final Answer:

   A DataFrame with columns 'apple', 'banana', 'orange' containing 1s and 0s indicating fruit presence -> Option B

4. Quick Check:

   OneHotEncoder output = separate 0/1 columns [OK]

Hint: OneHotEncoder creates one column per category with 0/1 [OK]

Common Mistakes:

• Expecting a single column with numbers
• Thinking OneHotEncoder fails on strings
• Assuming original column stays in encoded_df

4. Identify the error in this code snippet for label encoding and how to fix it:

from sklearn.preprocessing import LabelEncoder
data = pd.DataFrame({'color': ['red', 'blue', 'green']})
le = LabelEncoder()
encoded = le.fit_transform(data['color'])
print(encoded)

medium

A. data is a dict, not a DataFrame; convert to DataFrame before encoding

B. LabelEncoder cannot encode colors; use OneHotEncoder instead

C. fit_transform should be called on le, not on data['color']

D. Missing import for pandas; add 'import pandas as pd'

Solution

1. Step 1: Check imports and dependencies

   The code uses pd.DataFrame without importing pandas, causing NameError: name 'pd' is not defined.

2. Step 2: Fix by adding the import

   Add 'import pandas as pd' at the beginning, before defining data.

3. Final Answer:

   Missing import for pandas; add 'import pandas as pd' -> Option D

4. Quick Check:

   Missing pandas import for pd.DataFrame [OK]

Hint: Always import pandas before using pd.DataFrame [OK]

Common Mistakes:

• Not importing pandas before using pd.DataFrame
• Confusing LabelEncoder with OneHotEncoder
• Ignoring data structure requirements

5. You have a dataset with a column 'size' containing categories: 'small', 'medium', 'large', and 'medium'. You want to encode it so that 'small' = 0, 'medium' = 1, and 'large' = 2, but also keep track of the original order. Which encoding method and approach should you use?

hard

A. Use pandas Categorical with ordered=True and then convert to codes

B. Use OneHotEncoder to create separate columns for each size

C. Use LabelEncoder directly without ordering

D. Manually replace categories with numbers using a dictionary without ordering

Solution

1. Step 1: Understand the need to preserve order

   Since 'small' < 'medium' < 'large', encoding must respect this order.

2. Step 2: Use pandas Categorical with ordered=True

   Creating an ordered categorical allows converting categories to codes that reflect the order.

3. Step 3: Convert ordered categorical to codes

   Codes will be 0 for 'small', 1 for 'medium', 2 for 'large' as desired.

4. Final Answer:

   Use pandas Categorical with ordered=True and then convert to codes -> Option A

5. Quick Check:

   Ordered categorical preserves order in encoding [OK]

Hint: Use ordered pandas Categorical to keep category order [OK]

Common Mistakes:

• Using LabelEncoder which ignores order
• Using OneHotEncoder which loses order info
• Replacing categories without defining order