How to Clear GPU Memory in PyTorch Quickly and Safely

To clear GPU memory in PyTorch, use torch.cuda.empty_cache() which frees unused cached memory. For complete cleanup, delete variables holding tensors on GPU and call torch.cuda.empty_cache() to release memory back to the system.

Syntax

The main command to clear GPU memory cache in PyTorch is torch.cuda.empty_cache(). This function releases all unused cached memory held by PyTorch's CUDA allocator back to the GPU, making it available for other GPU tasks.

Note that this does not free memory occupied by active tensors or variables. To fully clear memory, you must delete or reassign those variables first.

python

import torch

torch.cuda.empty_cache()

Example

This example shows how to clear GPU memory by deleting tensors and then calling torch.cuda.empty_cache(). It prints GPU memory usage before and after clearing.

python

import torch

# Allocate a tensor on GPU
x = torch.randn(10000, 10000, device='cuda')

# Print memory allocated before clearing
print(f"Memory allocated before clearing: {torch.cuda.memory_allocated() / 1e6:.2f} MB")

# Delete the tensor to free memory
del x

# Clear cached memory
torch.cuda.empty_cache()

# Print memory allocated after clearing
print(f"Memory allocated after clearing: {torch.cuda.memory_allocated() / 1e6:.2f} MB")

Output

Memory allocated before clearing: 381.47 MB
Memory allocated after clearing: 0.00 MB

Common Pitfalls

• Not deleting variables: Calling torch.cuda.empty_cache() alone does not free memory used by existing tensors. You must delete or overwrite those tensors first.
• Confusing cache with allocated memory: empty_cache() frees cached memory, but memory_allocated() shows active memory usage. They are different.
• Expecting empty_cache() to reduce PyTorch memory footprint: It only releases unused cache to the GPU, not to the CPU or system.

python

import torch

# Wrong way: just empty cache without deleting tensors
x = torch.randn(10000, 10000, device='cuda')
torch.cuda.empty_cache()  # Does NOT free memory used by x
print(torch.cuda.memory_allocated() > 0)  # True

# Right way: delete tensor then empty cache
del x
torch.cuda.empty_cache()
print(torch.cuda.memory_allocated() == 0)  # Might be False if other tensors exist

Output

True
False

Quick Reference

Summary tips to manage GPU memory in PyTorch:

• Use torch.cuda.empty_cache() to release unused cached memory.
• Delete or overwrite GPU tensors to free active memory.
• Monitor memory with torch.cuda.memory_allocated() and torch.cuda.memory_reserved().
• Restart your Python session if memory leaks persist.

Key Takeaways

Use torch.cuda.empty_cache() to free unused GPU cached memory in PyTorch.

Always delete or overwrite GPU tensors before emptying cache to truly free memory.

torch.cuda.empty_cache() does not free memory held by active tensors.

Monitor GPU memory usage with torch.cuda.memory_allocated() to track active memory.

Restart your Python environment if GPU memory is not released due to leaks.