Optical flow concept in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Optical flow concept

Which metric matters for Optical Flow and WHY

Optical flow measures how pixels move between video frames. The key metric is End-Point Error (EPE). It shows the average distance between the predicted pixel movement and the true movement. A smaller EPE means the model tracks motion more accurately. This matters because good motion tracking helps in video analysis, robotics, and self-driving cars.

Confusion matrix or equivalent visualization

Optical flow is a regression task, not classification, so no confusion matrix applies. Instead, we use a visual comparison:

    Frame 1 pixels:  (x, y)
    Frame 2 pixels:  (x + u, y + v)  where (u, v) is the flow vector

    True flow vector: (u_true, v_true)
    Predicted flow vector: (u_pred, v_pred)

    End-Point Error (EPE) = sqrt((u_true - u_pred)**2 + (v_true - v_pred)**2)
    

Lower EPE means better prediction of pixel movement.

Precision vs Recall tradeoff (or equivalent) with examples

In optical flow, the tradeoff is between accuracy and smoothness of the flow field.

• High accuracy but low smoothness: The model predicts pixel movement very close to true values but may produce noisy or jagged flow, which can confuse downstream tasks.
• High smoothness but low accuracy: The flow looks smooth and clean but may miss small or fast movements, reducing usefulness.

Example: For a self-driving car, accurate detection of fast-moving objects is critical, so accuracy is prioritized. For video stabilization, smooth flow is more important.

What "good" vs "bad" metric values look like for Optical Flow

Good: EPE close to 0 pixels (e.g., < 1 pixel) means the predicted motion is very close to the true motion.

Bad: EPE greater than 5 pixels means the predicted motion is often far from the true motion, causing errors in motion understanding.

Visual checks also help: good flow fields show smooth, consistent arrows matching object movement; bad flow fields look noisy or random.

Common pitfalls in Optical Flow metrics

• Ignoring occlusions: Pixels that disappear or appear between frames can cause wrong error calculations.
• Using only average EPE: It can hide large errors in small regions; median or percentile errors give more insight.
• Overfitting to training data: Model may predict well on known scenes but fail on new videos.
• Data leakage: Using future frames or ground truth flow in training can give unrealistically low errors.

Self-check question

Your optical flow model has an average EPE of 0.5 pixels on test videos but shows noisy flow arrows in fast-moving areas. Is it good?

Answer: The low EPE suggests good average accuracy, but noisy flow in fast areas means the model struggles with quick motion. It may need improvement in those regions for reliable use.

Key Result

End-Point Error (EPE) is the key metric; lower EPE means better optical flow accuracy.