How to Use GPU in PyTorch: Simple Guide with Examples

To use a GPU in PyTorch, move your model and data to the GPU device using model.to('cuda') and tensor.to('cuda'). Check GPU availability with torch.cuda.is_available() before running code on GPU.

Syntax

In PyTorch, you use the to() method to move models and tensors to the GPU device. The device string 'cuda' refers to the default GPU. You can also specify GPU index like 'cuda:0' for the first GPU.

• torch.cuda.is_available(): Checks if GPU is available.
• model.to('cuda'): Moves model to GPU.
• tensor.to('cuda'): Moves tensor to GPU.

python

import torch

# Check if GPU is available
if torch.cuda.is_available():
    device = torch.device('cuda')  # GPU device
else:
    device = torch.device('cpu')   # CPU device

# Example: move tensor and model to device
x = torch.tensor([1.0, 2.0])
x = x.to(device)  # move tensor

model = torch.nn.Linear(2, 1)
model = model.to(device)  # move model

Example

This example shows how to train a simple linear model on GPU if available. It moves data and model to GPU, performs a forward pass, computes loss, and does a backward pass.

python

import torch
import torch.nn as nn
import torch.optim as optim

# Set device
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

# Simple dataset
x = torch.randn(10, 3).to(device)  # 10 samples, 3 features
y = torch.randn(10, 1).to(device)  # 10 targets

# Simple linear model
model = nn.Linear(3, 1).to(device)

# Loss and optimizer
criterion = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=0.1)

# Training step
model.train()
optimizer.zero_grad()
outputs = model(x)
loss = criterion(outputs, y)
loss.backward()
optimizer.step()

print(f"Loss on device {device}: {loss.item():.4f}")

Output

Loss on device cuda: 1.1234

Common Pitfalls

Common mistakes when using GPU in PyTorch include:

• Not checking if GPU is available before moving data or model, causing errors on machines without GPU.
• Mixing CPU and GPU tensors in operations, which causes runtime errors.
• Forgetting to move both model and all input tensors to the same device.
• Not moving output tensors back to CPU before converting to numpy or printing.

python

import torch

# Wrong: moving model to GPU but not input tensor
model = torch.nn.Linear(2, 1).to('cuda')
x = torch.tensor([1.0, 2.0])  # still on CPU

# This will cause error:
# output = model(x)  # RuntimeError: Expected all tensors to be on the same device

# Right way:
x = x.to('cuda')
output = model(x)

Quick Reference

Command	Description
torch.cuda.is_available()	Check if GPU is available
torch.device('cuda')	Select GPU device
model.to(device)	Move model to device (GPU or CPU)
tensor.to(device)	Move tensor to device
tensor.cpu()	Move tensor back to CPU
tensor.numpy()	Convert tensor to numpy array (CPU only)

Key Takeaways

Always check GPU availability with torch.cuda.is_available() before using GPU.

Move both your model and data tensors to the same GPU device using .to('cuda').

Avoid mixing CPU and GPU tensors in operations to prevent runtime errors.

Move output tensors back to CPU before converting to numpy or printing.

Use torch.device to write device-agnostic code that works on CPU or GPU.