How to Use .cuda() in PyTorch for GPU Acceleration

In PyTorch, use .cuda() to move tensors or models to the GPU for faster computation. For example, tensor.cuda() moves a tensor to the GPU, and model.cuda() moves the entire model to the GPU device.

Syntax

The .cuda() method is called on a PyTorch tensor or model to move it to the GPU device. You can optionally specify the GPU device index like .cuda(0) for the first GPU.

• tensor.cuda(): Moves a tensor to the default GPU.
• model.cuda(): Moves all model parameters to the GPU.
• tensor.cuda(device_id): Moves tensor to a specific GPU by index.

python

tensor = torch.tensor([1, 2, 3])
tensor_gpu = tensor.cuda()  # Moves to default GPU

model = MyModel()
model_gpu = model.cuda()  # Moves model to GPU

# Move tensor to GPU 1
tensor_gpu1 = tensor.cuda(1)

Example

This example shows how to create a tensor, move it to GPU using .cuda(), perform a simple operation, and print the device location.

python

import torch

# Check if GPU is available
if torch.cuda.is_available():
    # Create a tensor on CPU
    tensor_cpu = torch.tensor([1.0, 2.0, 3.0])
    print(f"Original device: {tensor_cpu.device}")

    # Move tensor to GPU
    tensor_gpu = tensor_cpu.cuda()
    print(f"After .cuda(): {tensor_gpu.device}")

    # Perform operation on GPU
    result = tensor_gpu * 2
    print(f"Result on device: {result.device}")
    print(result)
else:
    print("CUDA is not available on this machine.")

Output

Original device: cpu
After .cuda(): cuda:0
Result on device: cuda:0
tensor([2., 4., 6.], device='cuda:0')

Common Pitfalls

Common mistakes when using .cuda() include:

• Trying to move data to GPU when CUDA is not available, causing errors.
• Mixing CPU and GPU tensors in operations, which leads to runtime errors.
• Forgetting to move both model and input tensors to the same device.

Always check torch.cuda.is_available() before using .cuda().

python

import torch

# Wrong: Mixing CPU and GPU tensors
x_cpu = torch.tensor([1, 2])
x_gpu = x_cpu.cuda()
try:
    y = x_cpu + x_gpu  # This will raise an error
except RuntimeError as e:
    print(f"Error: {e}")

# Right: Move both tensors to GPU
x_cpu = torch.tensor([1, 2])
x_gpu = x_cpu.cuda()
y_gpu = x_cpu.cuda()
z = x_gpu + y_gpu
print(z)

Output

Error: expected device cpu but got device cuda:0
tensor([2, 4], device='cuda:0')

Quick Reference

Summary tips for using .cuda():

• Use .cuda() to move tensors or models to GPU.
• Check GPU availability with torch.cuda.is_available().
• Keep all tensors and models on the same device to avoid errors.
• You can specify GPU device index like .cuda(1) for multi-GPU setups.

Key Takeaways

Use .cuda() to move tensors or models to GPU for faster computation.

Always check torch.cuda.is_available() before using .cuda() to avoid errors.

Keep all tensors and models on the same device to prevent runtime errors.

You can specify the GPU device index with .cuda(device_id) for multi-GPU systems.

Mixing CPU and GPU tensors in operations causes errors; move all involved tensors to GPU.