GPU tensors (to, cuda) in PyTorch - Model Pipeline Trace

Model Pipeline - GPU tensors (to, cuda)

This pipeline shows how data moves from CPU memory to GPU memory using PyTorch tensors. It demonstrates creating tensors on CPU, transferring them to GPU for faster computation, performing a simple operation, and then moving results back to CPU.

Data Flow - 4 Stages

1Create tensor on CPU

N/A→Create a tensor with shape (3, 3) on CPU memory→3 rows x 3 columns

[[1, 2, 3], [4, 5, 6], [7, 8, 9]]

↓

2Transfer tensor to GPU

3 rows x 3 columns→Use .to('cuda') or .cuda() to move tensor to GPU memory→3 rows x 3 columns (on GPU)

Tensor with same values but stored on GPU device

↓

3Perform operation on GPU

3 rows x 3 columns (GPU tensor)→Add 10 to each element on GPU→3 rows x 3 columns (GPU tensor)

[[11, 12, 13], [14, 15, 16], [17, 18, 19]]

↓

4Transfer result back to CPU

3 rows x 3 columns (GPU tensor)→Use .to('cpu') to move tensor back to CPU memory→3 rows x 3 columns (CPU tensor)

[[11, 12, 13], [14, 15, 16], [17, 18, 19]]

Training Trace - Epoch by Epoch

Loss
0.5 |****
0.4 |***
0.3 |**
0.2 |*
0.1 | 
    +------------
     Epochs 1-5

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.45	0.65	Initial training on CPU, loss starts moderate, accuracy low
2	0.30	0.78	Training on GPU speeds up computation, loss decreases, accuracy improves
3	0.20	0.85	Continued GPU training shows steady improvement
4	0.15	0.90	Loss decreases further, accuracy reaches good level
5	0.12	0.92	Training converges well on GPU

Prediction Trace - 4 Layers

Layer 1: Input tensor on CPU

Layer 2: Transfer tensor to GPU

Layer 3: Add 5 to each element on GPU

Layer 4: Transfer result back to CPU

Model Quiz - 3 Questions

Why do we move tensors to GPU using .to('cuda') or .cuda()?

A To make tensors smaller in size

B To save memory on CPU

C To speed up calculations by using GPU hardware

D To convert tensors to a different data type

Key Insight

Moving tensors to GPU allows faster math operations, which helps training models converge quicker with better accuracy. The tensor shape stays the same; only the device changes. This is key for efficient deep learning.