Memory usage analysis in Pandas - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Memory usage of DataFrame columns

What is the output of the following code showing memory usage of each column in bytes?

Pandas

import pandas as pd
import numpy as np
df = pd.DataFrame({
    'A': np.arange(1000, dtype='int64'),
    'B': np.random.rand(1000),
    'C': ['text']*1000
})
mem_usage = df.memory_usage(deep=True)
print(mem_usage)

A

Index    128
A      8000
B      8000
C     24000
dtype: int64

B

Index    128
A      8000
B      8000
C     4000
dtype: int64

C

Index    128
A      8000
B      8000
C     32000
dtype: int64

D

Index    128
A      1000
B      8000
C     24000
dtype: int64

💡 Hint

Remember that strings with deep=True count the actual string memory, not just pointers.

The 'C' column contains 1000 identical strings 'text'. Each string takes about 24 bytes including overhead, so total is about 24000 bytes. Numeric columns use fixed sizes.

❓ data_output

intermediate

Total memory usage of DataFrame

What is the total memory usage in bytes of the DataFrame below, including index and deep memory?

Pandas

import pandas as pd
import numpy as np
df = pd.DataFrame({
    'X': np.random.randint(0, 100, 5000),
    'Y': ['a']*5000
})
total_mem = df.memory_usage(deep=True).sum()
print(total_mem)

A 90000

B 50000

C 100128

D 40000

💡 Hint

Consider index memory plus each column's memory with deep=True.

Index of 5000 int64 uses 40000 bytes, 'X' column 40000 bytes, 'Y' column 20128 bytes (strings). Total is 100128 bytes.

🔧 Debug

advanced

Why does memory usage not decrease after dropping a column?

Given the code below, why does the memory usage of df not decrease after dropping column 'B'?

Pandas

import pandas as pd
import numpy as np
df = pd.DataFrame({
    'A': np.arange(10000),
    'B': ['text']*10000
})
print(df.memory_usage(deep=True).sum())
df.drop('B', axis=1, inplace=True)
print(df.memory_usage(deep=True).sum())

A Because drop with inplace=True does not free memory immediately due to pandas internal caching.

B Because the 'B' column is still referenced elsewhere in the code, so memory is not freed.

C Because memory_usage(deep=True) always returns the same value regardless of columns.

D Because drop does not remove columns, it only hides them temporarily.

💡 Hint

Think about how pandas manages memory internally and when garbage collection happens.

pandas may keep memory allocated internally even after dropping columns due to caching and delayed garbage collection. Memory may not reduce immediately.

🧠 Conceptual

advanced

Effect of categorical dtype on memory usage

Which statement best describes the effect of converting a string column to category dtype on memory usage?

A It converts strings to integers but increases memory due to overhead.

B It increases memory usage because categories store extra metadata.

C It has no effect on memory usage but speeds up computations.

D It decreases memory usage by storing unique values once and using integer codes.

💡 Hint

Think about how categories store repeated values efficiently.

Converting to category stores unique strings once and replaces column with integer codes, reducing memory usage significantly.

🚀 Application

expert

Optimize memory usage of a large DataFrame

You have a DataFrame with 1 million rows and columns of various types. Which approach will most effectively reduce memory usage without losing data?

A Convert all integer columns to <code>int8</code> and all floats to <code>float16</code> regardless of value range.

B Convert object columns with few unique values to <code>category</code> and downcast numeric columns to smallest suitable subtype.

C Drop all columns with missing values to reduce memory usage.

D Convert all columns to string type to unify data and compress memory.

💡 Hint

Consider safe downcasting and categorical conversion for repeated strings.

Downcasting numeric columns to smallest subtype that fits data and converting low-cardinality object columns to category reduces memory without data loss. Blind conversion or dropping columns can cause data loss or inefficiency.