Why categorical type matters in Pandas - Challenge Your Understanding

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of memory usage with categorical vs object type

Consider a pandas DataFrame with a column of repeated string values. What will be the approximate memory usage difference when the column is converted to categorical type compared to keeping it as object type?

Pandas

import pandas as pd
import numpy as np

size = 100000
values = ['apple', 'banana', 'cherry', 'date']
data = pd.DataFrame({'fruit': np.random.choice(values, size)})

mem_before = data.memory_usage(deep=True).sum()
data['fruit'] = data['fruit'].astype('category')
mem_after = data.memory_usage(deep=True).sum()

print(mem_before, mem_after)

A Code raises a TypeError because categorical conversion is invalid

B mem_after is about the same as mem_before, no significant difference

C mem_after is significantly smaller than mem_before, roughly 5-10 times less memory used

D mem_after is larger than mem_before because categorical adds overhead

💡 Hint

Think about how categorical stores repeated values as codes instead of full strings.

Categorical type stores repeated values as integer codes and a separate categories list, which reduces memory usage significantly compared to storing full strings repeatedly.

❓ data_output

intermediate

Number of unique categories after conversion

Given a pandas Series with repeated string values, what will be the number of categories after converting it to categorical type?

Pandas

import pandas as pd
s = pd.Series(['red', 'blue', 'red', 'green', 'blue', 'blue'])
cat = s.astype('category')
print(len(cat.cat.categories))

A 6

B Raises AttributeError

C 1

D 3

💡 Hint

Count unique values in the original Series.

Categorical categories are the unique values in the Series. Here, 'red', 'blue', and 'green' are the unique values, so 3 categories.

🔧 Debug

advanced

Why does this code raise a TypeError?

This code tries to assign a new value to a categorical column but raises a TypeError. Why?

Pandas

import pandas as pd
s = pd.Series(['small', 'medium', 'large'], dtype='category')
s[0] = 'extra large'

A Because 'extra large' is not in the categories, assignment fails

B Because categorical columns are immutable and cannot be changed

C Because 'extra large' is a string and categorical only accepts integers

D Because pandas Series do not support item assignment

💡 Hint

Think about allowed values in categorical data.

You cannot assign a value not in the categories to a categorical Series. It raises a TypeError.

🚀 Application

advanced

Choosing categorical type for performance

You have a DataFrame with a column 'city' containing 1 million rows but only 50 unique city names. Which benefit will you get by converting 'city' to categorical type?

A Slower filtering but faster sorting on 'city'

B Reduced memory usage and faster groupby operations on 'city'

C No benefit because 50 unique values is too many for categorical

D Increased memory usage due to category overhead

💡 Hint

Think about how categorical stores repeated values and how groupby uses categories.

Categorical reduces memory by storing repeated values as codes and speeds up groupby by working on integer codes instead of strings.

🧠 Conceptual

expert

Why does sorting a categorical column with ordered=True differ from unordered?

What is the main difference in sorting behavior between a pandas categorical column with ordered=True and one with ordered=False?

A With ordered=True, sorting respects the category order; with ordered=False, sorting is by category codes

B With ordered=True, sorting is faster; with ordered=False, sorting raises an error

C With ordered=True, sorting is random; with ordered=False, sorting respects category order

D There is no difference; both sort alphabetically

💡 Hint

Think about what ordered categories mean for comparison.

Ordered categorical means categories have a defined order used in sorting. Unordered categorical sorts by the underlying codes.