How to Reduce Memory Usage in Python Efficiently

To reduce memory usage in Python, use generators instead of lists for large data, apply __slots__ in classes to avoid extra memory overhead, and use built-in modules like sys to monitor memory. Also, prefer data structures like collections.deque or array for efficient storage.

Syntax

Here are some common Python patterns to reduce memory usage:

• Generator expression: Use parentheses ( ) instead of brackets [ ] to create generators that produce items one by one.
• __slots__ in classes: Define __slots__ to limit instance attributes and save memory.
• Using sys.getsizeof(): Check the memory size of objects.

python

class MyClass:
    __slots__ = ['name', 'age']

    def __init__(self, name, age):
        self.name = name
        self.age = age

# Generator expression syntax
gen = (x * 2 for x in range(5))

# Using sys.getsizeof
import sys
size = sys.getsizeof(gen)

Example

This example shows how using a generator instead of a list saves memory when processing large data.

python

import sys

# Using list comprehension (uses more memory)
list_data = [x * 2 for x in range(1000000)]
print('List size:', sys.getsizeof(list_data), 'bytes')

# Using generator expression (uses less memory)
gen_data = (x * 2 for x in range(1000000))
print('Generator size:', sys.getsizeof(gen_data), 'bytes')

Output

List size: 8697456 bytes
Generator size: 112 bytes

Common Pitfalls

Common mistakes include using large lists when generators would suffice, not using __slots__ in classes with many instances, and holding references to unused objects causing memory leaks.

Also, using mutable default arguments can cause unexpected memory retention.

python

class PersonWrong:
    def __init__(self, name):
        self.name = name

class PersonRight:
    __slots__ = ['name']
    def __init__(self, name):
        self.name = name

import sys
p1 = PersonWrong('Alice')
p2 = PersonRight('Bob')
print('Without __slots__:', sys.getsizeof(p1))
print('With __slots__:', sys.getsizeof(p2))

Output

Without __slots__: 56
With __slots__: 48

Quick Reference

• Use generators to handle large data streams efficiently.
• Apply __slots__ in classes to reduce attribute storage overhead.
• Use sys.getsizeof() to measure object memory size.
• Prefer collections.deque or array.array for memory-efficient data structures.
• Delete unused variables and use gc.collect() to free memory.

Key Takeaways

Use generators instead of lists to save memory when processing large data.

Define __slots__ in classes to reduce memory overhead for many instances.

Measure object size with sys.getsizeof() to identify memory-heavy objects.

Choose memory-efficient data structures like collections.deque or array.array.

Avoid holding references to unused objects to prevent memory leaks.