Categorical data type optimization in Data Analysis Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of memory usage after converting to categorical

What is the output of the following Python code that optimizes a DataFrame column by converting it to a categorical type?

Data Analysis Python

import pandas as pd
import numpy as np

# Create a DataFrame with 100000 rows
np.random.seed(0)
data = pd.DataFrame({
    'color': np.random.choice(['red', 'green', 'blue'], size=100000)
})

# Memory usage before conversion
mem_before = data.memory_usage(deep=True).sum()

# Convert 'color' column to categorical
data['color'] = data['color'].astype('category')

# Memory usage after conversion
mem_after = data.memory_usage(deep=True).sum()

print(round(mem_before), round(mem_after))

A 240000 240000

B 240000 80000

C 300000 300000

D 80000 240000

💡 Hint

Think about how categorical data stores repeated values efficiently compared to strings.

Converting a string column to categorical reduces memory usage significantly because it stores unique values once and uses integer codes for each row.

❓ data_output

intermediate

Number of unique categories after optimization

Given a DataFrame with a 'city' column converted to categorical, what is the number of unique categories stored?

Data Analysis Python

import pandas as pd

cities = ['New York', 'Los Angeles', 'Chicago', 'Houston', 'Phoenix', 'New York', 'Chicago']
data = pd.DataFrame({'city': cities})
data['city'] = data['city'].astype('category')

num_categories = len(data['city'].cat.categories)
print(num_categories)

A 4

B 7

C 6

D 5

💡 Hint

Count unique city names ignoring duplicates.

The unique cities are New York, Los Angeles, Chicago, Houston, and Phoenix, totaling 5 categories.

🔧 Debug

advanced

Identify the error in categorical conversion code

What error does the following code produce?

Data Analysis Python

import pandas as pd

values = ['apple', 'banana', 'apple', 'orange']
data = pd.DataFrame({'fruit': values})
data['fruit'] = data['fruit'].astype('category', categories=['apple', 'banana'])

A TypeError: astype() got an unexpected keyword argument 'categories'

B ValueError: Cannot set categories with astype()

C KeyError: 'categories'

D No error, runs successfully

💡 Hint

Check the correct way to specify categories when converting to categorical.

The astype() method does not accept a 'categories' argument. Categories must be set using pd.Categorical or cat.set_categories().

🚀 Application

advanced

Choosing categorical dtype for large dataset

You have a dataset with a column 'status' containing 3 unique values repeated millions of times. Which approach optimizes memory best?

A Keep 'status' as object dtype (string)

B Convert 'status' to integer dtype without mapping

C Convert 'status' to category dtype

D Convert 'status' to float dtype

💡 Hint

Think about how categorical dtype stores repeated values efficiently.

Category dtype stores unique values once and uses integer codes internally, saving memory for repeated values.

🧠 Conceptual

expert

Effect of categorical ordering on sorting performance

How does setting the ordered=True parameter in a categorical column affect sorting performance in pandas?

A Sorting is faster because pandas uses the category order for comparisons

B Sorting is slower because ordered categories require extra checks

C Sorting raises an error if categories are ordered

D Sorting performance is unchanged by ordered parameter

💡 Hint

Consider how ordered categories allow direct integer comparison during sorting.

When categories are ordered, pandas can sort by comparing integer codes directly, which is faster than sorting strings.

Practice

1.

What is the main benefit of converting a DataFrame column to category type in pandas?

easy

A. It reduces memory usage by storing unique values once

B. It automatically sorts the column values alphabetically

C. It converts numeric values to strings

D. It removes missing values from the column

Solution

1. Step 1: Understand what categorical type does

   The categorical type stores unique values only once and uses codes internally, saving memory.

2. Step 2: Compare with other options

   Sorting, converting to strings, or removing missing values are not automatic effects of categorical conversion.

3. Final Answer:

   It reduces memory usage by storing unique values once -> Option A

4. Quick Check:

   Categorical type = Memory saving [OK]

Hint: Categorical stores unique values once to save memory [OK]

Common Mistakes:

• Thinking it sorts values automatically
• Confusing with string conversion
• Assuming it removes missing data

2.

Which of the following is the correct syntax to convert a pandas DataFrame column df['color'] to categorical type?

import pandas as pd
df = pd.DataFrame({'color': ['red', 'blue', 'red', 'green']})

easy

A. df['color'] = df.astype('category')

B. df['color'] = df['color'].astype('category')

C. df['color'] = pd.category(df['color'])

D. df['color'] = df['color'].to_category()

Solution

1. Step 1: Identify correct method to convert column type

   The astype('category') method converts a Series to categorical type.

2. Step 2: Check other options for errors

   df['color'] = df.astype('category') tries to convert whole DataFrame, not column. df['color'] = pd.category(df['color']) uses non-existent pd.category(). df['color'] = df['color'].to_category() uses wrong method name.

3. Final Answer:

   df['color'] = df['color'].astype('category') -> Option B

4. Quick Check:

   astype('category') = Correct syntax [OK]

Hint: Use astype('category') on the specific column [OK]

Common Mistakes:

• Applying astype to whole DataFrame instead of column
• Using non-existent pd.category() function
• Using wrong method like to_category()

3.

What will be the output of the following code?

import pandas as pd

df = pd.DataFrame({'fruit': ['apple', 'banana', 'apple', 'orange', 'banana']})
df['fruit'] = df['fruit'].astype('category')
print(df['fruit'].cat.categories)

medium

A. Index(['apple', 'banana', 'orange'], dtype='object')

B. ['apple', 'banana', 'apple', 'orange', 'banana']

C. ['apple', 'banana', 'orange', 'apple', 'banana']

D. Error: 'cat' attribute not found

Solution

1. Step 1: Convert column to categorical and check categories

   After conversion, cat.categories shows unique sorted categories.

2. Step 2: Identify unique values in sorted order

   Unique fruits are 'apple', 'banana', 'orange' in alphabetical order.

3. Final Answer:

   Index(['apple', 'banana', 'orange'], dtype='object') -> Option A

4. Quick Check:

   cat.categories = unique sorted values [OK]

Hint: cat.categories shows unique sorted categories [OK]

Common Mistakes:

• Expecting original list instead of unique categories
• Confusing categories with values
• Thinking cat attribute is missing

4.

Identify the error in this code snippet that tries to convert a column to categorical:

import pandas as pd
df = pd.DataFrame({'size': ['S', 'M', 'L', 'M']})
df['size'] = df['size'].astype(category)

medium

A. astype() cannot convert to categorical

B. DataFrame creation syntax is wrong

C. Missing quotes around 'category' in astype()

D. Column 'size' does not exist

Solution

1. Step 1: Check astype() argument

   The argument to astype() must be a string 'category', not a bare word.

2. Step 2: Verify other parts of code

   DataFrame creation and column name are correct; astype() supports 'category'.

3. Final Answer:

   Missing quotes around 'category' in astype() -> Option C

4. Quick Check:

   Quotes needed for dtype string [OK]

Hint: Always put quotes around dtype names in astype() [OK]

Common Mistakes:

• Forgetting quotes around 'category'
• Assuming astype() can't convert to category
• Mistaking column name errors

5.

You have a DataFrame with a column 'city' containing 1 million rows but only 5 unique city names. Which approach optimizes memory and speeds up grouping by city?

hard

A. Use df.groupby('city') without any conversion

B. Convert 'city' column to string type before grouping

C. Drop duplicates from 'city' column before grouping

D. Convert 'city' column to categorical type before grouping

Solution

1. Step 1: Understand memory and speed benefits of categorical

   Converting to categorical stores unique values once, reducing memory and speeding grouping.

2. Step 2: Evaluate other options

   String type uses more memory; dropping duplicates loses data; grouping without conversion is slower.

3. Final Answer:

   Convert 'city' column to categorical type before grouping -> Option D

4. Quick Check:

   Categorical = Memory + speed optimization [OK]

Hint: Convert to categorical before grouping large repeated data [OK]

Common Mistakes:

• Using string type wastes memory
• Dropping duplicates loses data
• Grouping without conversion is slower