Memory savings with categoricals in Pandas - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to convert the 'color' column to a categorical type.

Pandas

df['color'] = df['color'].[1]()

Drag options to blanks, or click blank then click option'

Aastype('category')

Bto_numeric()

Cstr()

Dcopy()

Attempts:

3 left

💡 Hint

Common Mistakes

Using to_numeric() which converts to numbers, not categories.

Using str() which converts to strings, not categories.

Using astype('category') converts the column to a categorical type, which saves memory.

2fill in blank

medium

Complete the code to check the memory usage of the DataFrame.

Pandas

memory_before = df.memory_usage([1]=True).sum()

Drag options to blanks, or click blank then click option'

Adetailed

Bshallow

Cdeep

Dverbose

Attempts:

3 left

💡 Hint

Common Mistakes

Using shallow which is not a valid parameter.

Using verbose which is for printing info, not memory calculation.

The deep=True option gives a more accurate memory usage including object data.

3fill in blank

hard

Fix the error in the code to convert the 'city' column to categorical.

Pandas

df['city'] = df['city'].[1]('category')

Drag options to blanks, or click blank then click option'

Aconvert

Bastype

Cto_type

Dchange_type

Attempts:

3 left

💡 Hint

Common Mistakes

Using non-existent methods like convert() or to_type().

The correct method to cast a pandas column type is astype().

4fill in blank

hard

Complete the code to calculate the memory usage after the conversions.

Pandas

memory_after = df.[1]([2]=True).[3]()

Drag options to blanks, or click blank then click option'

Amemory_usage

Bdeep

Csum

Dinfo

Attempts:

3 left

💡 Hint

Common Mistakes

Using info() which shows summary info, not memory.

Forgetting to call sum() on the Series.

Use df.memory_usage(deep=True).sum() to get the total memory usage after converting columns to categorical.

5fill in blank

hard

Fill the blanks to compute the percentage of memory saved.

Pandas

savings_pct = round( ([1] - memory_after) / [1] * [2], [3] )

Drag options to blanks, or click blank then click option'

Amemory_before

B100

C2

Dmemory_after

Attempts:

3 left

💡 Hint

Common Mistakes

Using memory_after instead of memory_before in the denominator.

Forgetting to multiply by 100 for percentage.

Wrong number of decimals in round().

The formula is round( (memory_before - memory_after) / memory_before * 100, 2 ) to get percentage saved rounded to 2 decimals.

Practice

1.

What is the main benefit of converting a pandas column to category type?

easy

A. It sorts the data automatically

B. It saves memory by storing repeated values efficiently

C. It converts numbers to strings

D. It removes missing values

Solution

1. Step 1: Understand what category type does

   The category type stores repeated text values once and uses codes internally.

2. Step 2: Identify the main benefit

   This reduces memory usage because repeated strings are not stored multiple times.

3. Final Answer:

   It saves memory by storing repeated values efficiently -> Option B

4. Quick Check:

   Memory savings = A [OK]

Hint: Remember: categories store repeated text once [OK]

Common Mistakes:

• Thinking it sorts data automatically
• Confusing category with string conversion
• Assuming it removes missing values

2.

Which of the following is the correct way to convert a pandas DataFrame column df['color'] to a categorical type?

import pandas as pd
df = pd.DataFrame({'color': ['red', 'blue', 'red', 'green']})

easy

A. df['color'] = df['color'].astype('category')

B. df['color'] = df['color'].to_category()

C. df['color'] = pd.category(df['color'])

D. df['color'] = df['color'].convert('category')

Solution

1. Step 1: Recall pandas syntax for type conversion

   Use astype('category') to convert a column to categorical type.

2. Step 2: Check each option

   Only df['color'] = df['color'].astype('category') uses the correct astype method with 'category' string.

3. Final Answer:

   df['color'] = df['color'].astype('category') -> Option A

4. Quick Check:

   astype('category') = A [OK]

Hint: Use astype('category') to convert columns [OK]

Common Mistakes:

• Using non-existent methods like to_category()
• Trying to call pd.category() which does not exist
• Using convert() instead of astype()

3.

What will be the output of the following code?

import pandas as pd
import sys

df = pd.DataFrame({'fruit': ['apple', 'banana', 'apple', 'banana', 'cherry']})
mem_before = df['fruit'].memory_usage()
df['fruit'] = df['fruit'].astype('category')
mem_after = df['fruit'].memory_usage()
print(mem_before, mem_after)

medium

A. 112 80

B. 80 80

C. 80 112

D. 112 112

Solution

1. Step 1: Calculate memory before conversion

   Strings take more memory; here memory_usage() returns 112 bytes before conversion.

2. Step 2: Calculate memory after conversion to category

   Category stores codes and unique values separately, reducing memory to 80 bytes.

3. Final Answer:

   112 80 -> Option A

4. Quick Check:

   Memory before > after = C [OK]

Hint: Memory decreases after astype('category') [OK]

Common Mistakes:

• Assuming memory stays the same
• Thinking memory increases after conversion
• Confusing order of printed values

4.

Identify the error in the following code that tries to convert a column to categorical and save memory:

import pandas as pd
df = pd.DataFrame({'city': ['NY', 'LA', 'NY', 'SF']})
df['city'] = df['city'].astype(category)

medium

A. DataFrame creation syntax is wrong

B. astype cannot convert to category type

C. Missing quotes around 'category' in astype

D. Column 'city' does not exist

Solution

1. Step 1: Check astype argument

   The argument to astype must be a string 'category', but here category is unquoted, causing a NameError.

2. Step 2: Verify other parts

   DataFrame creation and column name are correct; astype supports 'category'.

3. Final Answer:

   Missing quotes around 'category' in astype -> Option C

4. Quick Check:

   Quotes needed for 'category' string [OK]

Hint: Always quote 'category' in astype('category') [OK]

Common Mistakes:

• Forgetting quotes around 'category'
• Thinking astype can't convert to category
• Assuming column name is wrong

5.

You have a DataFrame with a column df['status'] containing 1 million rows with only 3 unique values: 'new', 'in_progress', 'done'. How can you reduce memory usage effectively?

hard

A. Replace values with integers manually without category

B. Convert df['status'] to string type

C. Drop the df['status'] column

D. Convert df['status'] to category type

Solution

1. Step 1: Understand the data characteristics

   With only 3 unique values repeated 1 million times, storing as category saves memory by storing unique values once.

2. Step 2: Compare options

   Converting to string keeps large memory; dropping column loses data; manual integer replacement is error-prone and less efficient.

3. Final Answer:

   Convert df['status'] to category type -> Option D

4. Quick Check:

   Few unique values + large rows = category [OK]

Hint: Use category for few unique values in large data [OK]

Common Mistakes:

• Using string type which uses more memory
• Dropping column losing important data
• Manual integer replacement without category benefits