Converting to categorical in Pandas - Step-by-Step Execution

Concept Flow - Converting to categorical

Start with DataFrame

↓

Select column to convert

↓

Apply pd.Categorical()

↓

Column type changes to 'category'

↓

Use categorical benefits: less memory, faster ops

↓

End

Start with a DataFrame, pick a column, convert it to categorical type, then use the benefits of categories.

Execution Sample

Pandas

import pandas as pd

df = pd.DataFrame({'color': ['red', 'blue', 'green', 'blue', 'red']})
df['color'] = pd.Categorical(df['color'])
print(df['color'])

This code converts the 'color' column in the DataFrame to categorical type.

Execution Table

Step	Action	Input/Variable	Result/Output
1	Create DataFrame	{'color': ['red', 'blue', 'green', 'blue', 'red']}	DataFrame with 'color' column as object type
2	Select 'color' column	df['color']	Series with values ['red', 'blue', 'green', 'blue', 'red'] and dtype object
3	Convert to categorical	pd.Categorical(df['color'])	Categorical dtype with categories ['blue', 'green', 'red']
4	Assign back to df['color']	df['color'] = pd.Categorical(df['color'])	'color' column dtype changes to 'category'
5	Print df['color']	df['color']	Shows categorical data with categories and values
6	Check dtype	df['color'].dtype	category

💡 Conversion complete, 'color' column is now categorical type.

Variable Tracker

Variable	Start	After Step 1	After Step 3	After Step 4	Final
df	None	{'color': ['red', 'blue', 'green', 'blue', 'red']}	Same DataFrame, 'color' still object dtype	Same DataFrame, 'color' changed to categorical dtype	DataFrame with 'color' column as categorical
df['color']	None	Series of strings (object)	Series of strings (object)	Categorical dtype with categories ['blue', 'green', 'red']	Categorical dtype Series

Key Moments - 3 Insights

Why does the 'color' column dtype change after assignment?

Because pd.Categorical creates a categorical object, and assigning it back replaces the original column with this new categorical type, as shown in execution_table step 4.

Are the original string values changed when converting to categorical?

No, the values remain the same but are stored more efficiently with categories, as seen in execution_table step 5 where values print the same but dtype is 'category'.

What are the categories created during conversion?

Categories are unique sorted values from the column, here ['blue', 'green', 'red'], shown in execution_table step 3.

Visual Quiz - 3 Questions

Look at the execution_table, what is the dtype of df['color'] after step 3?

A object

B int

C category

D string

💡 Hint

Step 3 creates a Categorical (dtype 'category') but does not assign it to df['color'] yet; the dtype of df['color'] remains 'object' until step 4.

At which step does the 'color' column in df change to categorical dtype?

A Step 3

B Step 4

C Step 2

D Step 5

💡 Hint

Look at the 'Action' and 'Result/Output' columns in execution_table for when assignment happens.

If we skip assigning pd.Categorical back to df['color'], what will be the dtype of df['color']?

A category

B int

C object

D float

💡 Hint

Refer to variable_tracker for df['color'] before and after assignment.

Concept Snapshot

Convert a DataFrame column to categorical with:
  df['col'] = pd.Categorical(df['col'])
This changes dtype to 'category', saving memory and speeding up operations.
Categories are unique sorted values.
Assign back to keep the change.
Use categorical for repeated string data.

Full Transcript

We start with a DataFrame containing a column of strings. We select that column and convert it to a categorical type using pandas' pd.Categorical function. This creates a categorical object with unique categories. We assign this back to the DataFrame column, changing its dtype to 'category'. This conversion keeps the original values but stores them more efficiently. We can verify the change by printing the column and checking its dtype. This process helps save memory and can speed up operations on repeated string data.

Practice

1. What is the main benefit of converting a text column to a categorical type in pandas?

easy

A. It removes duplicate rows from the DataFrame.

B. It changes the text to numbers automatically.

C. It saves memory and speeds up analysis.

D. It sorts the DataFrame by that column.

Solution

1. Step 1: Understand what categorical type does

   Converting text to categorical stores repeated values efficiently, saving memory.

2. Step 2: Recognize impact on analysis speed

   Operations on categorical data are faster because pandas uses codes internally.

3. Final Answer:

   It saves memory and speeds up analysis. -> Option C

4. Quick Check:

   Converting to categorical = Save memory and speed [OK]

Hint: Categorical = less memory + faster analysis [OK]

Common Mistakes:

• Thinking it changes text to numbers visibly
• Assuming it sorts data automatically
• Confusing with removing duplicates

2. Which of the following is the correct syntax to convert a pandas DataFrame column df['color'] to categorical?

easy

A. df['color'] = df['color'].astype('category')

B. df['color'] = pd.to_categorical(df['color'])

C. df['color'] = df['color'].convert('category')

D. df['color'] = df['color'].category()

Solution

1. Step 1: Recall pandas method for type conversion

   The correct method is astype('category') to convert column type.

2. Step 2: Check each option

   df['color'] = df['color'].astype('category') uses astype('category'), which is correct. Others are invalid pandas syntax.

3. Final Answer:

   df['color'] = df['color'].astype('category') -> Option A

4. Quick Check:

   Use astype('category') to convert [OK]

Hint: Use astype('category') to convert columns [OK]

Common Mistakes:

• Using non-existent methods like convert() or category()
• Trying pd.to_categorical which is not a pandas function
• Missing quotes around 'category'

3. Given the code:

import pandas as pd
df = pd.DataFrame({'size': ['S', 'M', 'L', 'M', 'S']})
df['size'] = pd.Categorical(df['size'], categories=['S', 'M', 'L'], ordered=True)
print(df['size'].dtype)

What will be the output?

medium

A. category

B. object

C. string

D. int64

Solution

1. Step 1: Understand pd.Categorical usage

   Using pd.Categorical converts the column to categorical dtype with order.

2. Step 2: Check dtype after conversion

   The dtype of the column becomes category, not object or string.

3. Final Answer:

   category -> Option A

4. Quick Check:

   pd.Categorical sets dtype to category [OK]

Hint: pd.Categorical sets dtype to category [OK]

Common Mistakes:

• Expecting dtype to remain object
• Confusing category with string dtype
• Thinking dtype becomes int64

4. What is wrong with this code snippet?

import pandas as pd
df = pd.DataFrame({'grade': ['A', 'B', 'C', 'B', 'A']})
df['grade'] = pd.Categorical(df['grade'], categories=['A', 'B', 'C'])
df['grade'] = df['grade'].astype('category', ordered=True)

medium

A. DataFrame column cannot be converted twice.

B. astype() does not accept ordered=True argument.

C. pd.Categorical requires categories to be sorted alphabetically.

D. Missing import statement for pandas.

Solution

1. Step 1: Check astype() method parameters

   The astype() method does not accept an ordered parameter.

2. Step 2: Identify correct way to set ordered

   Ordering must be set when creating the categorical with pd.Categorical(..., ordered=True), not via astype.

3. Final Answer:

   astype() does not accept ordered=True argument. -> Option B

4. Quick Check:

   ordered=True only in pd.Categorical, not astype [OK]

Hint: ordered=True only works in pd.Categorical, not astype [OK]

Common Mistakes:

• Trying to set ordered in astype()
• Assuming categories must be sorted alphabetically
• Thinking multiple conversions cause error

5. You have a DataFrame df with a column 'status' containing repeated text values: 'low', 'medium', 'high'. You want to convert it to a categorical type with the order low < medium < high, and then get the counts of each category in order. Which code snippet correctly does this?

hard

A. df['status'] = df['status'].astype('category', ordered=True)
print(df['status'].value_counts(sort=False))

B. df['status'] = df['status'].astype('category')
print(df['status'].value_counts())

C. df['status'] = pd.Categorical(df['status'])
print(df['status'].value_counts(sort=True))

D. df['status'] = pd.Categorical(df['status'], categories=['low', 'medium', 'high'], ordered=True)
print(df['status'].value_counts(sort=False))

Solution

1. Step 1: Convert with ordered categories

   Use pd.Categorical with categories=['low', 'medium', 'high'] and ordered=True to set order.

2. Step 2: Get counts without sorting

   Use value_counts(sort=False) to get counts in category order.

3. Final Answer:

   df['status'] = pd.Categorical(df['status'], categories=['low', 'medium', 'high'], ordered=True) print(df['status'].value_counts(sort=False)) -> Option D

4. Quick Check:

   pd.Categorical + ordered + value_counts(sort=False) = correct [OK]

Hint: Use pd.Categorical with ordered and value_counts(sort=False) [OK]

Common Mistakes:

• Using astype('category') without order
• Not setting ordered=True for category order
• Using value_counts() without sort=False loses order