Categorical data type optimization in Data Analysis Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is a categorical data type in data analysis?

A categorical data type is a way to store data that has a limited set of possible values, like colors or categories, which helps save memory and speeds up analysis.

beginner

Why should you convert string columns with repeated values to categorical type?

Because categorical type stores each unique value only once and uses codes for the data, reducing memory use and making operations faster.

intermediate

How does pandas represent categorical data internally?

Pandas stores categorical data as integer codes that point to a list of unique categories, which saves space compared to storing full strings repeatedly.

beginner

What is a common method to convert a column to categorical in pandas?

Use the pandas function: df['column'] = df['column'].astype('category') to convert the column to categorical type.

intermediate

What is a potential downside of using categorical data type?

If the column has many unique values (high cardinality), converting to categorical might not save memory and can add overhead.

What does converting a column to categorical type mainly help with?

A Increasing the number of unique values

B Reducing memory usage

C Changing data to numbers only

D Removing missing values

Categorical type reduces memory by storing unique values once and using codes.

Which pandas method converts a column to categorical?

A df['col'].astype('category')

B df['col'].to_numeric()

C df['col'].fillna()

D df['col'].unique()

astype('category') changes the column type to categorical.

What is stored internally for categorical data in pandas?

A Boolean values

B Full strings repeated

C Integer codes and categories list

D Only numbers

Pandas stores integer codes pointing to unique categories.

When might categorical type NOT save memory?

A When there are many unique values

B When data is numeric

C When data has missing values

D When data is already numeric

High cardinality means many unique values, reducing memory savings.

Which of these is a benefit of categorical data type?

A Converting numbers to strings

B More precise floating point calculations

C Automatic data cleaning

D Faster comparisons and grouping

Categorical data speeds up operations like grouping and comparisons.

Explain what categorical data type is and why it is useful in data analysis.

Think about how repeated strings can be stored more efficiently.

Describe how to convert a string column to categorical in pandas and when you should do it.

Focus on the pandas method and the type of data that benefits most.

Practice

1.

What is the main benefit of converting a DataFrame column to category type in pandas?

easy

A. It reduces memory usage by storing unique values once

B. It automatically sorts the column values alphabetically

C. It converts numeric values to strings

D. It removes missing values from the column

Solution

1. Step 1: Understand what categorical type does

   The categorical type stores unique values only once and uses codes internally, saving memory.

2. Step 2: Compare with other options

   Sorting, converting to strings, or removing missing values are not automatic effects of categorical conversion.

3. Final Answer:

   It reduces memory usage by storing unique values once -> Option A

4. Quick Check:

   Categorical type = Memory saving [OK]

Hint: Categorical stores unique values once to save memory [OK]

Common Mistakes:

• Thinking it sorts values automatically
• Confusing with string conversion
• Assuming it removes missing data

2.

Which of the following is the correct syntax to convert a pandas DataFrame column df['color'] to categorical type?

import pandas as pd
df = pd.DataFrame({'color': ['red', 'blue', 'red', 'green']})

easy

A. df['color'] = df.astype('category')

B. df['color'] = df['color'].astype('category')

C. df['color'] = pd.category(df['color'])

D. df['color'] = df['color'].to_category()

Solution

1. Step 1: Identify correct method to convert column type

   The astype('category') method converts a Series to categorical type.

2. Step 2: Check other options for errors

   df['color'] = df.astype('category') tries to convert whole DataFrame, not column. df['color'] = pd.category(df['color']) uses non-existent pd.category(). df['color'] = df['color'].to_category() uses wrong method name.

3. Final Answer:

   df['color'] = df['color'].astype('category') -> Option B

4. Quick Check:

   astype('category') = Correct syntax [OK]

Hint: Use astype('category') on the specific column [OK]

Common Mistakes:

• Applying astype to whole DataFrame instead of column
• Using non-existent pd.category() function
• Using wrong method like to_category()

3.

What will be the output of the following code?

import pandas as pd

df = pd.DataFrame({'fruit': ['apple', 'banana', 'apple', 'orange', 'banana']})
df['fruit'] = df['fruit'].astype('category')
print(df['fruit'].cat.categories)

medium

A. Index(['apple', 'banana', 'orange'], dtype='object')

B. ['apple', 'banana', 'apple', 'orange', 'banana']

C. ['apple', 'banana', 'orange', 'apple', 'banana']

D. Error: 'cat' attribute not found

Solution

1. Step 1: Convert column to categorical and check categories

   After conversion, cat.categories shows unique sorted categories.

2. Step 2: Identify unique values in sorted order

   Unique fruits are 'apple', 'banana', 'orange' in alphabetical order.

3. Final Answer:

   Index(['apple', 'banana', 'orange'], dtype='object') -> Option A

4. Quick Check:

   cat.categories = unique sorted values [OK]

Hint: cat.categories shows unique sorted categories [OK]

Common Mistakes:

• Expecting original list instead of unique categories
• Confusing categories with values
• Thinking cat attribute is missing

4.

Identify the error in this code snippet that tries to convert a column to categorical:

import pandas as pd
df = pd.DataFrame({'size': ['S', 'M', 'L', 'M']})
df['size'] = df['size'].astype(category)

medium

A. astype() cannot convert to categorical

B. DataFrame creation syntax is wrong

C. Missing quotes around 'category' in astype()

D. Column 'size' does not exist

Solution

1. Step 1: Check astype() argument

   The argument to astype() must be a string 'category', not a bare word.

2. Step 2: Verify other parts of code

   DataFrame creation and column name are correct; astype() supports 'category'.

3. Final Answer:

   Missing quotes around 'category' in astype() -> Option C

4. Quick Check:

   Quotes needed for dtype string [OK]

Hint: Always put quotes around dtype names in astype() [OK]

Common Mistakes:

• Forgetting quotes around 'category'
• Assuming astype() can't convert to category
• Mistaking column name errors

5.

You have a DataFrame with a column 'city' containing 1 million rows but only 5 unique city names. Which approach optimizes memory and speeds up grouping by city?

hard

A. Use df.groupby('city') without any conversion

B. Convert 'city' column to string type before grouping

C. Drop duplicates from 'city' column before grouping

D. Convert 'city' column to categorical type before grouping

Solution

1. Step 1: Understand memory and speed benefits of categorical

   Converting to categorical stores unique values once, reducing memory and speeding grouping.

2. Step 2: Evaluate other options

   String type uses more memory; dropping duplicates loses data; grouping without conversion is slower.

3. Final Answer:

   Convert 'city' column to categorical type before grouping -> Option D

4. Quick Check:

   Categorical = Memory + speed optimization [OK]

Hint: Convert to categorical before grouping large repeated data [OK]

Common Mistakes:

• Using string type wastes memory
• Dropping duplicates loses data
• Grouping without conversion is slower