Optical flow concept in Computer Vision - Model Pipeline Trace

Model Pipeline - Optical flow concept

Optical flow estimates how pixels move between two video frames. It helps computers understand motion by tracking changes in pixel positions over time.

Data Flow - 5 Stages

1Input frames

2 frames x 480 rows x 640 columns x 3 channels→Load two consecutive video frames→2 frames x 480 rows x 640 columns x 3 channels

Frame 1 and Frame 2 from a video showing a moving car

↓

2Grayscale conversion

2 frames x 480 rows x 640 columns x 3 channels→Convert RGB frames to grayscale to simplify processing→2 frames x 480 rows x 640 columns x 1 channel

Grayscale images highlighting brightness changes

↓

3Compute pixel differences

2 frames x 480 rows x 640 columns x 1 channel→Calculate brightness changes between frames→1 frame x 480 rows x 640 columns x 1 channel

Difference image showing moving edges

↓

4Estimate flow vectors

1 frame x 480 rows x 640 columns x 1 channel→Calculate motion vectors (horizontal and vertical) for each pixel→1 frame x 480 rows x 640 columns x 2 channels

Flow field showing direction and speed of pixel movement

↓

5Post-processing

1 frame x 480 rows x 640 columns x 2 channels→Smooth and filter flow vectors to reduce noise→1 frame x 480 rows x 640 columns x 2 channels

Cleaned flow vectors ready for visualization or analysis

Training Trace - Epoch by Epoch

Loss
0.5 |****
0.4 |*** 
0.3 |**  
0.2 |*   
0.1 |    
     1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.45	0.60	Initial training with high error and moderate accuracy
2	0.32	0.75	Loss decreased, accuracy improved as model learns motion patterns
3	0.25	0.82	Better flow estimation, smoother vector predictions
4	0.20	0.87	Model converging with consistent improvement
5	0.17	0.90	Final epoch shows low loss and high accuracy

Prediction Trace - 4 Layers

Layer 1: Input frames

Layer 2: Compute pixel differences

Layer 3: Estimate flow vectors

Layer 4: Post-processing

Model Quiz - 3 Questions

What does the optical flow algorithm primarily estimate?

A The brightness of a single pixel

B The movement of pixels between two frames

C The color changes in a single image

D The shape of objects in one frame

Key Insight

Optical flow helps machines understand motion by tracking pixel changes between frames. Converting to grayscale simplifies calculations, and training improves the model's ability to predict smooth and accurate motion vectors.