Motion detection basics in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Motion detection basics

Which metric matters for motion detection and WHY

In motion detection, the key metrics are Recall and Precision. Recall tells us how many actual motions were correctly found. Precision tells us how many detected motions were real. We want high recall to catch all movements, and high precision to avoid false alarms. The F1 score balances these two. Accuracy is less useful because most frames have no motion, so a model guessing "no motion" often looks good but is not helpful.

Confusion matrix example for motion detection

      | Predicted Motion | Predicted No Motion |
      |------------------|---------------------|
      | True Motion: 80  | False Negative: 20  |
      | False Positive: 10| True No Motion: 890 |

      Total frames = 1000

      Precision = TP / (TP + FP) = 80 / (80 + 10) = 0.89
      Recall = TP / (TP + FN) = 80 / (80 + 20) = 0.80
      F1 Score = 2 * (0.89 * 0.80) / (0.89 + 0.80) ≈ 0.84
    

Precision vs Recall tradeoff with examples

If the motion detector is set to be very sensitive, it finds almost all motions (high recall) but also triggers many false alarms (low precision). This is like a security camera that alerts you too often, causing annoyance.

If it is set to be strict, it only alerts when very sure (high precision) but might miss some motions (low recall). This could miss important events.

Choosing the right balance depends on the use case. For home security, high recall is often more important to not miss intruders. For wildlife monitoring, high precision might be preferred to avoid wasting time on false alerts.

What good vs bad metric values look like for motion detection

• Good: Precision and recall both above 0.85, meaning most motions are detected and false alarms are low.
• Bad: Precision below 0.5 means many false alarms; recall below 0.5 means many motions missed.
• Accuracy can be misleadingly high (e.g., 95%) if the model mostly predicts no motion because most frames have no motion.

Common pitfalls in motion detection metrics

• Accuracy paradox: High accuracy but poor detection because most frames have no motion.
• Data leakage: Using future frames or labels during training can inflate metrics falsely.
• Overfitting: Model works well on training video but fails on new scenes.
• Ignoring class imbalance: Motion frames are rare, so metrics must focus on recall and precision, not just accuracy.

Self-check question

Your motion detection model has 98% accuracy but only 12% recall on motion frames. Is it good for production? Why or why not?

Answer: No, it is not good. The model misses 88% of actual motions (low recall), which defeats the purpose of motion detection. The high accuracy is misleading because most frames have no motion, so guessing "no motion" often looks good but is useless.

Key Result

Recall and precision are key for motion detection; accuracy alone can be misleading due to class imbalance.