Memory usage analysis in Data Analysis Python

Introduction

Memory usage analysis helps you understand how much computer memory your data and programs use. This is important to keep your work fast and avoid crashes.

When working with large datasets and you want to check if they fit in your computer's memory.

When your program is running slowly and you suspect it uses too much memory.

Before sharing data or models to ensure they are not too big to handle.

When optimizing code to use less memory and run faster.

Syntax

Data Analysis Python

dataframe.memory_usage(deep=True)

# or for total memory in MB
memory_in_mb = dataframe.memory_usage(deep=True).sum() / (1024 ** 2)

memory_usage() is a method for pandas DataFrames to check memory used by each column.

The deep=True option gives a more accurate size by including object types like strings.

Examples

This shows memory used by each column and the index in bytes.

Data Analysis Python

import pandas as pd

data = {'name': ['Anna', 'Bob', 'Cara'], 'age': [25, 30, 22]}
df = pd.DataFrame(data)

print(df.memory_usage(deep=True))

This calculates total memory used by the DataFrame in megabytes for easier reading.

Data Analysis Python

total_mem = df.memory_usage(deep=True).sum() / (1024 ** 2)
print(f"Total memory usage: {total_mem:.4f} MB")

Sample Program

This program creates a small table with different types of data. It then shows how much memory each column uses and the total memory in megabytes.

Data Analysis Python

import pandas as pd

# Create a sample DataFrame with different data types
data = {
    'id': range(1, 6),
    'name': ['Alice', 'Bob', 'Charlie', 'David', 'Eva'],
    'score': [85.5, 90.0, 78.0, 92.5, 88.0],
    'passed': [True, True, False, True, True]
}
df = pd.DataFrame(data)

# Check memory usage of each column including object types
mem_usage = df.memory_usage(deep=True)
print("Memory usage per column (in bytes):")
print(mem_usage)

# Calculate total memory usage in megabytes
total_mem_mb = mem_usage.sum() / (1024 ** 2)
print(f"\nTotal memory usage of DataFrame: {total_mem_mb:.6f} MB")

Important Notes

Memory usage depends on data types; numeric types use less memory than strings.

Using deep=True is important to get accurate size for columns with text data.

Checking memory helps you decide if you need to reduce data size or use more memory-efficient types.

Summary

Memory usage analysis shows how much memory your data uses.

Use memory_usage(deep=True) on pandas DataFrames to get detailed info.

Knowing memory use helps keep your programs fast and stable.