Adding and removing categories in Pandas

Introduction

Categories help organize data with fixed possible values. Adding or removing categories lets you update these possible values as your data changes.

You get new types of data not in your original categories.

You want to clean up categories by removing unused or wrong ones.

You want to prepare data for analysis by defining all possible groups.

You need to fix category lists after merging data from different sources.

Syntax

Pandas

Series.cat.add_categories(new_categories)
Series.cat.remove_categories(categories_to_remove)

Use add_categories() to add new category values.

Use remove_categories() to remove unwanted category values.

Examples

This adds 'cherry' as a new category to the Series.

Pandas

import pandas as pd
s = pd.Series(['apple', 'banana'], dtype='category')
s = s.cat.add_categories(['cherry'])
print(s.cat.categories)

This removes 'banana' from the categories.

Pandas

import pandas as pd
s = pd.Series(['apple', 'banana', 'cherry'], dtype='category')
s = s.cat.remove_categories(['banana'])
print(s.cat.categories)

Sample Program

This program shows how to add and remove categories from a pandas Series. It starts with 'apple' and 'banana' categories, adds 'cherry', then removes 'banana'.

Pandas

import pandas as pd

# Create a Series with categories
fruits = pd.Series(['apple', 'banana', 'apple', 'banana'], dtype='category')
print('Original categories:', fruits.cat.categories)

# Add a new category 'cherry'
fruits = fruits.cat.add_categories(['cherry'])
print('After adding category:', fruits.cat.categories)

# Remove the category 'banana'
fruits = fruits.cat.remove_categories(['banana'])
print('After removing category:', fruits.cat.categories)

# Show the Series values
print('Series values:')
print(fruits)

Important Notes

Removing a category replaces its values with NaN in the Series.

Adding categories does not change existing data, only the list of allowed categories.

Summary

Categories define fixed possible values in data.

Use add_categories() to add new possible values.

Use remove_categories() to remove unwanted values, which replaces them with NaN.

Practice

1. What does the add_categories() method do in pandas categorical data?

easy

A. It converts a column to a categorical type.

B. It removes categories from a categorical column.

C. It adds new possible categories to an existing categorical column.

D. It renames existing categories in a categorical column.

Solution

1. Step 1: Understand the purpose of add_categories()

   This method is used to add new possible categories to a pandas categorical column without changing existing data.

2. Step 2: Differentiate from other methods

   Unlike remove_categories() which removes categories, add_categories() only adds new ones.

3. Final Answer:

   It adds new possible categories to an existing categorical column. -> Option C

4. Quick Check:

   add_categories() adds categories [OK]

Hint: Remember: add_categories adds new options, doesn't remove [OK]

Common Mistakes:

• Confusing add_categories with remove_categories
• Thinking it changes existing data values
• Assuming it renames categories

2. Which of the following is the correct syntax to remove categories from a pandas categorical column named df['color']?

easy

A. df['color'].remove_categories(['red', 'blue'])

B. df['color'].drop_categories(['red', 'blue'])

C. df['color'].delete_categories(['red', 'blue'])

D. df['color'].pop_categories(['red', 'blue'])

Solution

1. Step 1: Recall the correct method name

   The pandas method to remove categories is remove_categories().

2. Step 2: Match the syntax with the method

   df['color'].remove_categories(['red', 'blue']) uses remove_categories() correctly on the categorical column.

3. Final Answer:

   df['color'].remove_categories(['red', 'blue']) -> Option A

4. Quick Check:

   Use remove_categories() to remove categories [OK]

Hint: Remember method name is remove_categories, not drop or delete [OK]

Common Mistakes:

• Using non-existent methods like drop_categories
• Confusing remove_categories with add_categories
• Trying to remove categories with pop or delete

3. Given the code:

import pandas as pd
cat = pd.Categorical(['apple', 'banana', 'apple'], categories=['apple', 'banana', 'cherry'])
cat2 = cat.remove_categories(['banana'])
print(cat2)

What will be the output?

medium

A. [apple, NaN, apple]
Categories (2, object): [apple, cherry]

B. [apple, banana, apple]
Categories (2, object): [apple, cherry]

C. [apple, NaN, apple]
Categories (3, object): [apple, banana, cherry]

D. [apple, banana, apple]
Categories (3, object): [apple, banana, cherry]

Solution

1. Step 1: Understand remove_categories effect on data

   Removing 'banana' category replaces all 'banana' values with NaN and removes 'banana' from categories.

2. Step 2: Analyze the printed output

   Values become ['apple', NaN, 'apple'] and categories become ['apple', 'cherry'] with length 2.

3. Final Answer:

   [apple, NaN, apple] Categories (2, object): [apple, cherry] -> Option A

4. Quick Check:

   Removed category replaced by NaN [OK]

Hint: Removing category replaces values with NaN [OK]

Common Mistakes:

• Expecting removed category values to stay unchanged
• Thinking categories length remains same
• Confusing categories list after removal

4. Consider this code snippet:

import pandas as pd
cat = pd.Categorical(['dog', 'cat', 'bird'], categories=['dog', 'cat', 'bird'])
cat = cat.add_categories('fish')
cat = cat.remove_categories('cat')

What is the error in this code?

medium

A. Categorical cannot have more than 3 categories.

B. add_categories cannot add a single category.

C. You cannot chain add_categories and remove_categories.

D. remove_categories expects a list, not a string.

Solution

1. Step 1: Check method argument types

   remove_categories() requires a list or iterable of categories to remove, not a single string.

2. Step 2: Identify the error cause

   Passing 'cat' as a string causes a TypeError or unexpected behavior.

3. Final Answer:

   remove_categories expects a list, not a string. -> Option D

4. Quick Check:

   Pass list to remove_categories [OK]

Hint: Always pass list to remove_categories, not single string [OK]

Common Mistakes:

• Passing single string instead of list to remove_categories
• Thinking add_categories can't add one item
• Assuming chaining methods is invalid

5. You have a pandas categorical column df['status'] with categories ['new', 'in_progress', 'done']. You want to add a new category 'archived' and remove 'new' without losing data for other rows. Which sequence of commands achieves this correctly?

hard

A. df['status'] = df['status'].remove_categories(['new'])
df['status'].add_categories(['archived'])

B. df['status'] = df['status'].add_categories(['archived'])
df['status'] = df['status'].remove_categories(['new'])

C. df['status'].remove_categories(['new'])
df['status'].add_categories(['archived'])

D. df['status'].add_categories('archived')
df['status'].remove_categories('new')

Solution

1. Step 1: Add new category before removing

   Adding 'archived' first ensures it exists as a valid category before removal.

2. Step 2: Remove 'new' category properly

   Removing 'new' replaces its values with NaN, so do this after adding new category and assign back to column.

3. Step 3: Assign results back to the column

   Both methods return new categorical objects, so assign back to df['status'].

4. Final Answer:

   df['status'] = df['status'].add_categories(['archived']) df['status'] = df['status'].remove_categories(['new']) -> Option B

5. Quick Check:

   Add then remove, assign back [OK]

Hint: Add categories first, then remove, assign back [OK]

Common Mistakes:

• Removing category before adding new one
• Not assigning back to the DataFrame column
• Passing string instead of list to methods