Ordered categories in Pandas - Step-by-Step Execution

Concept Flow - Ordered categories

Create categorical data

↓

Define category order

↓

Assign ordered categories

↓

Use categories in analysis

↓

Compare or sort using order

We start with data, define an order for categories, assign this order, then use it to compare or sort data.

Execution Sample

Pandas

import pandas as pd
cats = pd.Categorical(['low', 'medium', 'high', 'medium'],
                      categories=['low', 'medium', 'high'],
                      ordered=True)
print(cats > 'low')

This code creates an ordered categorical variable and compares each value to 'low'.

Execution Table

Step	Action	Input/Condition	Result/Output
1	Create categorical data	['low', 'medium', 'high', 'medium']	Categorical object with values ['low', 'medium', 'high', 'medium']
2	Define categories and order	categories=['low', 'medium', 'high'], ordered=True	Categories set with order: low < medium < high
3	Assign ordered categories	Assign to variable 'cats'	cats is ordered categorical with given categories
4	Compare cats > 'low'	Compare each element to 'low'	[False, True, True, True]
5	Print result	Output comparison array	[False, True, True, True]

💡 Comparison done element-wise using the defined order, execution ends after print.

Variable Tracker

Variable	Start	After 1	After 2	After 3	After 4	Final
cats	None	['low', 'medium', 'high', 'medium']	Ordered categories set	Assigned to cats variable	Compared to 'low'	Ordered categorical with given categories

Key Moments - 3 Insights

Why does comparing categories like 'cats > "low"' work?

Because 'cats' is an ordered categorical with a defined order low < medium < high, pandas can compare each element to 'low' using this order (see execution_table step 4).

What happens if the categories are not ordered?

Without ordering, comparisons like 'cats > "low"' will raise an error because pandas cannot determine which category is greater (not shown in execution_table but important to know).

Why do we need to specify categories explicitly?

Specifying categories and their order ensures pandas knows the full set and the order, even if some categories don't appear in data (see execution_table step 2).

Visual Quiz - 3 Questions

Look at the execution table at step 4, what is the comparison result of 'cats > "low"' for the first element?

A False

B True

C Error

D None

💡 Hint

Check the 'Result/Output' column in step 4 of the execution_table.

At which step in the execution table is the order of categories defined?

A Step 1

B Step 3

C Step 2

D Step 4

💡 Hint

Look for where 'Categories set with order' is mentioned in the execution_table.

If we remove 'ordered=True' from the code, what will happen when comparing 'cats > "low"'?

A Comparison returns all True

B Raises an error

C Comparison returns all False

D Comparison ignores order and works

💡 Hint

Recall the key moment about comparisons without ordering.

Concept Snapshot

Ordered categories in pandas:
- Use pd.Categorical(data, categories=[...], ordered=True)
- Defines a fixed order for categories
- Enables comparisons and sorting
- Without ordered=True, comparisons fail
- Useful for data with natural order like ratings

Full Transcript

This example shows how to create ordered categorical data in pandas. We start by creating a categorical variable with values like 'low', 'medium', and 'high'. We explicitly define the categories and set ordered=True to tell pandas the order low < medium < high. This allows us to compare each category value to another, for example checking which are greater than 'low'. The comparison returns a boolean array showing True for 'medium' and 'high' and False for 'low'. This ordering is important because without it, pandas cannot compare categories. Specifying categories also ensures pandas knows all possible categories and their order, even if some are missing in the data. This technique is useful for data science tasks where categories have a natural order, like ratings or sizes.

Practice

1. What does setting ordered=True do when creating a pandas categorical variable?

easy

A. It converts categories to numeric values automatically.

B. It sorts the categories alphabetically.

C. It defines a natural order for the categories, allowing comparisons.

D. It removes duplicate categories from the list.

Solution

1. Step 1: Understand the role of ordered categories

   Ordered categories allow pandas to know the sequence of categories, so it can compare them meaningfully.

2. Step 2: Effect of ordered=True

   Setting ordered=True tells pandas to treat the categories as having a specific order, enabling operations like greater than or less than.

3. Final Answer:

   It defines a natural order for the categories, allowing comparisons. -> Option C

4. Quick Check:

   ordered=True means categories have order [OK]

Hint: Ordered=True means categories can be compared by order [OK]

Common Mistakes:

• Thinking it sorts categories automatically
• Assuming it converts categories to numbers
• Believing it removes duplicates

2. Which of the following is the correct way to create an ordered categorical variable in pandas with categories 'low', 'medium', 'high' in that order?

easy

A. pd.Categorical(['low', 'medium', 'high'], categories=['low', 'medium', 'high'], ordered=True)

B. pd.Categorical(['low', 'tiny', 'high'], categories=['low', 'medium', 'high'], ordered=True)

C. pd.Categorical(['low', 'medium', 'high'], ordered=True, categories=['low', 'medium', 'high'])

D. pd.Categorical(['low', 'medium', 'high'], ordered=False, categories=['low', 'medium', 'high'])

Solution

1. Step 1: Check parameter order and names

   pd.Categorical takes data first, then categories, then ordered flag as keyword arguments.

2. Step 2: Identify correct syntax

   pd.Categorical(['low', 'medium', 'high'], categories=['low', 'medium', 'high'], ordered=True) correctly passes data, categories list, and ordered=True in correct order and names.

3. Final Answer:

   pd.Categorical(['low', 'medium', 'high'], categories=['low', 'medium', 'high'], ordered=True) -> Option A

4. Quick Check:

   Correct syntax uses categories= and ordered=True [OK]

Hint: Use categories= first, then ordered=True keyword [OK]

Common Mistakes:

• Passing ordered=True before categories
• Using ordered=False for ordered categories
• Not matching categories list with data

3. What will be the output of this code?

import pandas as pd
cats = pd.Categorical(['medium', 'low', 'high'], categories=['low', 'medium', 'high'], ordered=True)
print(cats > 'low')

medium

A. [False, True, True]

B. [False, False, True]

C. [True, True, False]

D. [True, False, True]

Solution

1. Step 1: Understand the categories and order

   Categories are ordered as low < medium < high.

2. Step 2: Compare each element to 'low'

   'medium' > 'low' is True, 'low' > 'low' is False, 'high' > 'low' is True.

3. Final Answer:

   [True, False, True] -> Option D

4. Quick Check:

   medium and high are greater than low [OK]

Hint: Compare each category to 'low' using order [OK]

Common Mistakes:

• Confusing category order
• Assuming string comparison instead of categorical
• Mixing True/False positions

4. The following code raises an error. What is the cause?

import pandas as pd
cats = pd.Categorical(['low', 'medium', 'high'], categories=['low', 'medium'], ordered=True)
print(cats > 'medium')

medium

A. Comparison operator '>' is not supported for categorical data.

B. The categories list is missing 'high', causing invalid comparison.

C. Ordered=True cannot be used with categories parameter.

D. The data list has duplicate values.

Solution

1. Step 1: Check categories vs data

   Data contains 'high' but categories only include 'low' and 'medium'.

2. Step 2: Understand effect on comparison

   'high' is not in categories, so comparison with 'medium' fails causing error.

3. Final Answer:

   The categories list is missing 'high', causing invalid comparison. -> Option B

4. Quick Check:

   Data must match categories exactly [OK]

Hint: Ensure all data values are in categories list [OK]

Common Mistakes:

• Thinking ordered=True disallows categories
• Believing '>' is unsupported for ordered categories
• Ignoring missing categories in list

5. You have a DataFrame with a column 'size' containing values 'small', 'medium', 'large', and 'extra large'. You want to sort the DataFrame by 'size' in the natural order small < medium < large < extra large. Which code correctly achieves this?

hard

A. df['size'] = pd.Categorical(df['size'], categories=['small', 'medium', 'large', 'extra large'], ordered=True)
df.sort_values('size')

B. df['size'] = pd.Categorical(df['size'], ordered=True)
df.sort_values('size')

C. df['size'] = pd.Categorical(df['size'], categories=['extra large', 'large', 'medium', 'small'], ordered=True)
df.sort_values('size')

D. df.sort_values('size') # No need to set categories

Solution

1. Step 1: Define ordered categories with correct order

   Set categories=['small', 'medium', 'large', 'extra large'] with ordered=True to define natural order.

2. Step 2: Sort DataFrame by the ordered categorical column

   Use df.sort_values('size') to sort according to the defined order.

3. Final Answer:

   df['size'] = pd.Categorical(df['size'], categories=['small', 'medium', 'large', 'extra large'], ordered=True) df.sort_values('size') -> Option A

4. Quick Check:

   Set ordered categories then sort [OK]

Hint: Set ordered categories with correct order before sorting [OK]

Common Mistakes:

• Not specifying categories order
• Using wrong category order
• Trying to sort without ordered categories