Why edge deployment enables real-time CV in Computer Vision - Why Metrics Matter

Metrics & Evaluation - Why edge deployment enables real-time CV

Which metric matters for this concept and WHY

For real-time computer vision (CV) on edge devices, latency is the key metric. Latency measures how fast the model processes each image or video frame. Low latency means the system responds quickly, which is essential for real-time tasks like object detection in self-driving cars or live video analysis. Accuracy is also important but must be balanced with speed to ensure timely decisions.

Confusion matrix or equivalent visualization (ASCII)

Example confusion matrix for a real-time object detector on edge device:

          Predicted
          Cat  Dog  None
Actual Cat  45   3    2
       Dog   4   40   6
       None  1    2   97

Total samples = 200

- True Positives (TP): Correct detections (e.g., Cat predicted as Cat = 45)
- False Positives (FP): Wrong detections (e.g., Cat predicted as Dog = 3)
- False Negatives (FN): Missed detections (e.g., Cat predicted as None = 2)
- True Negatives (TN): Correctly ignored (e.g., None predicted as None = 97)

Precision vs Recall tradeoff with concrete examples

In edge CV, precision and recall tradeoff affects real-time performance:

• High Precision: Few false alarms. Important when false alerts cause costly actions, like stopping a robot unnecessarily.
• High Recall: Few misses. Important when missing an object is dangerous, like not detecting a pedestrian.

Example: A security camera on edge should have high recall to catch all intruders, even if some false alarms happen. But too many false alarms (low precision) can waste resources.

What "good" vs "bad" metric values look like for this use case

• Good: Latency under 50 milliseconds per frame, precision and recall above 85%, and F1 score above 0.85. This means fast and accurate detection suitable for real-time use.
• Bad: Latency over 200 milliseconds per frame, precision or recall below 60%, causing slow or unreliable responses that fail real-time needs.

Metrics pitfalls

• Ignoring latency: A very accurate model that is too slow is useless for real-time edge CV.
• Accuracy paradox: High accuracy on imbalanced data can be misleading if rare but important classes are missed.
• Data leakage: Training on data too similar to test data inflates metrics but fails in real-world edge deployment.
• Overfitting: Model performs well on training but poorly on new edge data, hurting recall and precision.

Self-check question

Your edge CV model has 98% accuracy but only 12% recall on detecting pedestrians. Is it good for real-time safety? Why or why not?

Answer: No, it is not good. Despite high accuracy, the very low recall means the model misses most pedestrians. This is dangerous for real-time safety because missing pedestrians can cause accidents. High recall is critical here.

Key Result

Low latency combined with balanced precision and recall is essential for effective real-time computer vision on edge devices.