Memory usage analysis in Data Analysis Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Memory usage of pandas DataFrame columns

What is the output of the following code showing memory usage of each column in a pandas DataFrame?

Data Analysis Python

import pandas as pd
import numpy as np
df = pd.DataFrame({
    'A': np.arange(1000),
    'B': np.random.rand(1000),
    'C': ['text']*1000
})
mem_usage = df.memory_usage(deep=True)
print(mem_usage)

A

Index    128
A       1000
B       8000
C      40000
dtype: int64

B

Index    128
A       8000
B       8000
C      32000
dtype: int64

C

Index    128
A       8000
B       8000
C      61000
dtype: int64

D

Index    128
A       8000
B       1000
C      40000
dtype: int64

💡 Hint

Remember that integers and floats take 8 bytes each, and strings with repeated values can have shared memory but deep=True counts actual string memory.

The integer column 'A' has 1000 values * 8 bytes = 8000 bytes. The float column 'B' also has 8000 bytes. The string column 'C' has 1000 identical strings, each taking about 61 bytes including overhead, totaling 61000 bytes. The index memory is 128 bytes.

❓ data_output

intermediate

Memory usage reduction by changing data types

Given a DataFrame with 1 million integers stored as int64, what is the memory usage after converting the column to int8?

Data Analysis Python

import pandas as pd
import numpy as np
df = pd.DataFrame({'numbers': np.arange(1000000)})
mem_before = df.memory_usage(deep=True).numbers

df['numbers'] = df['numbers'].astype('int8')
mem_after = df.memory_usage(deep=True).numbers
print(mem_before, mem_after)

A 8000000 8000000

B 8000000 1000000

C 1000000 8000000

D 1000000 1000000

💡 Hint

int64 uses 8 bytes per value, int8 uses 1 byte per value.

Original memory is 1,000,000 values * 8 bytes = 8,000,000 bytes. After converting to int8, memory is 1,000,000 * 1 byte = 1,000,000 bytes.

🔧 Debug

advanced

Identify the cause of high memory usage in a DataFrame

Why does this DataFrame use more memory than expected?

import pandas as pd
df = pd.DataFrame({
    'col': ['apple', 'banana', 'apple', 'banana'] * 250000
})
print(df.memory_usage(deep=True))

A Because the DataFrame uses int64 for strings which is inefficient.

B Because the DataFrame index is very large and uses most memory.

C Because the DataFrame has many missing values increasing memory.

D Because the strings are stored as object dtype without using categorical type, so each string is stored separately.

💡 Hint

Check how pandas stores repeated strings in object dtype.

Object dtype stores each string separately, even if repeated. Using categorical dtype would reduce memory by storing unique strings once.

🚀 Application

advanced

Visualizing memory usage of DataFrame columns

Which code snippet correctly creates a bar plot showing memory usage of each column in a DataFrame df?

A

import matplotlib.pyplot as plt
mem = df.memory_usage(deep=True)
mem.plot(kind='bar')
plt.show()

B

import matplotlib.pyplot as plt
mem = df.memory_usage(deep=True)
plt.plot(mem)
plt.show()

C

import matplotlib.pyplot as plt
mem = df.memory_usage()
plt.bar(mem.index, mem.values)
plt.show()

D

import matplotlib.pyplot as plt
mem = df.memory_usage(deep=True)
plt.scatter(mem.index, mem.values)
plt.show()

💡 Hint

Use pandas plotting with kind='bar' for bar charts.

Option A uses pandas Series plot with kind='bar' which correctly shows a bar chart of memory usage per column. Option A misses deep=True and uses plt.bar directly but may not label axes well. Option A uses line plot which is less clear. Option A uses scatter plot which is not suitable here.

🧠 Conceptual

expert

Understanding memory usage impact of categorical data

Which statement best explains why converting a string column with many repeated values to categorical dtype reduces memory usage?

A Categorical dtype stores the unique strings once and replaces column values with integer codes, reducing memory.

B Categorical dtype compresses strings using zip compression internally, saving memory.

C Categorical dtype converts strings to fixed-length byte arrays, which always use less memory.

D Categorical dtype stores strings as pointers to the original strings, increasing memory usage.

💡 Hint

Think about how categorical data stores repeated values.

Categorical dtype stores unique values once and replaces each value with a small integer code, which uses less memory than storing full strings repeatedly.

Practice

1. What does the memory_usage(deep=True) method in pandas do?

easy

A. It counts the number of rows in the DataFrame.

B. It deletes unused memory from the DataFrame.

C. It compresses the DataFrame to use less memory.

D. It shows detailed memory usage of each column including object types.

Solution

1. Step 1: Understand memory_usage method

   The memory_usage method reports memory used by each column in a DataFrame.

2. Step 2: Effect of deep=True parameter

   Using deep=True makes it include memory used by object types like strings, giving detailed info.

3. Final Answer:

   It shows detailed memory usage of each column including object types. -> Option D

4. Quick Check:

   memory_usage(deep=True) = detailed memory info [OK]

Hint: Remember deep=True reveals full memory use including strings [OK]

Common Mistakes:

• Thinking it deletes or compresses data
• Confusing it with counting rows
• Ignoring deep=True effect

2. Which of the following is the correct syntax to get memory usage of a pandas DataFrame df including deep analysis?

easy

A. memory_usage(df, deep=True)

B. df.memoryusage(deep=True)

C. df.memory_usage(deep=True)

D. df.get_memory_usage(deep=True)

Solution

1. Step 1: Recall pandas method name

   The correct method is memory_usage with underscore, called on DataFrame.

2. Step 2: Check method call syntax

   It is called as df.memory_usage(deep=True). Other options have wrong method names or syntax.

3. Final Answer:

   df.memory_usage(deep=True) -> Option C

4. Quick Check:

   Correct method name and call = df.memory_usage(deep=True) [OK]

Hint: Use exact method name: memory_usage with underscore [OK]

Common Mistakes:

• Misspelling method as memoryusage
• Using function style instead of method
• Adding get_ prefix incorrectly

3. Given the following code, what is the output?

import pandas as pd
df = pd.DataFrame({
  'A': [1, 2, 3],
  'B': ['x', 'y', 'z']
})
print(df.memory_usage(deep=True))

medium

A. Error because deep=True is invalid

B. Index memory + int64 column + object column memory sizes

C. Total memory size as a single number

D. Only memory size of numeric columns

Solution

1. Step 1: Understand DataFrame columns

   Column 'A' is integers, 'B' is strings (object type), and index is default integer index.

2. Step 2: memory_usage(deep=True) output

   This method returns a Series showing memory used by index and each column separately, including deep memory for object column 'B'.

3. Final Answer:

   Index memory + int64 column + object column memory sizes -> Option B

4. Quick Check:

   memory_usage(deep=True) lists memory per column [OK]

Hint: memory_usage(deep=True) shows memory per column including index [OK]

Common Mistakes:

• Expecting a single total number output
• Ignoring index memory
• Thinking deep=True causes error

4. What is wrong with this code snippet?

import pandas as pd
df = pd.DataFrame({'col': ['a', 'b', 'c']})
print(df.memory_usage(deep=False))

medium

A. It does not show full memory usage of string data.

B. The method memory_usage does not exist.

C. deep=False causes a syntax error.

D. DataFrame must have numeric columns to use memory_usage.

Solution

1. Step 1: Check memory_usage with deep=False

   By default or with deep=False, memory_usage does not count full memory used by object columns like strings.

2. Step 2: Effect on output

   This means the reported memory for 'col' will be underestimated, missing actual string memory.

3. Final Answer:

   It does not show full memory usage of string data. -> Option A

4. Quick Check:

   deep=False skips detailed object memory [OK]

Hint: Use deep=True to get full string memory usage [OK]

Common Mistakes:

• Thinking memory_usage method is missing
• Assuming deep=False causes error
• Believing numeric columns are required

5. You have a large DataFrame with many string columns. You want to reduce memory usage without losing data. Which approach helps most based on memory usage analysis?

hard

A. Convert string columns to categorical type after checking memory usage.

B. Drop all string columns to save memory.

C. Use memory_usage(deep=False) to estimate memory and ignore strings.

D. Convert all columns to float type regardless of original data.

Solution

1. Step 1: Analyze memory usage of string columns

   Using memory_usage(deep=True) shows string columns use a lot of memory.

2. Step 2: Convert strings to categorical

   Changing string columns to categorical reduces memory by storing unique values once.

3. Step 3: Avoid data loss

   Converting to categorical keeps data but uses less memory, unlike dropping or wrong type conversion.

4. Final Answer:

   Convert string columns to categorical type after checking memory usage. -> Option A

5. Quick Check:

   Convert strings to categorical for memory saving [OK]

Hint: Convert strings to categorical to save memory without losing data [OK]

Common Mistakes:

• Dropping important columns blindly
• Ignoring deep=True memory insights
• Converting all columns to float causing errors