Raspberry Pi deployment in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Raspberry Pi deployment

Which metric matters for Raspberry Pi deployment and WHY

When deploying computer vision models on a Raspberry Pi, the key metrics to watch are inference speed and accuracy. Inference speed tells us how fast the model can make predictions on the device, which is important because Raspberry Pi has limited computing power. Accuracy shows how well the model recognizes images or objects. We want a balance: a model fast enough to run smoothly on the Pi but still accurate enough to be useful.

Confusion matrix example for Raspberry Pi model

Suppose the model detects cats and dogs. Here is a confusion matrix from test data:

      | Predicted Cat | Predicted Dog |
      |--------------|---------------|
      | True Cat: 40 | False Dog: 5  |
      | False Cat: 3 | True Dog: 52  |
    

Total samples = 40 + 5 + 3 + 52 = 100

From this, we calculate:

• Precision for Cat = TP / (TP + FP) = 40 / (40 + 3) = 0.93
• Recall for Cat = TP / (TP + FN) = 40 / (40 + 5) = 0.89

Precision vs Recall tradeoff with Raspberry Pi examples

Imagine a home security camera on Raspberry Pi detecting intruders:

• High Precision: Few false alarms. The camera rarely mistakes a pet for an intruder. This avoids annoying alerts.
• High Recall: The camera catches almost every real intruder. Missing one is risky.

On Raspberry Pi, if the model is too complex to run fast, you might lower recall to keep speed. Or if safety is critical, you accept slower speed for higher recall.

Good vs Bad metric values for Raspberry Pi deployment

Good: Accuracy above 85%, inference time under 1 second per image, precision and recall balanced around 0.9. This means the model is both fast and reliable.

Bad: Accuracy below 70%, inference time over 3 seconds, or very low recall (below 0.5). This means the model is either too slow or misses too many objects, making it unusable on Raspberry Pi.

Common pitfalls in Raspberry Pi deployment metrics

• Accuracy paradox: High accuracy but poor recall if data is unbalanced (e.g., mostly background images).
• Data leakage: Testing on images very similar to training can give false high accuracy.
• Overfitting: Model works well on test data but slow or inaccurate on real Raspberry Pi images.
• Ignoring latency: A model with great accuracy but too slow to run on Raspberry Pi is not practical.

Self-check question

Your Raspberry Pi model has 98% accuracy but only 12% recall on detecting intruders. Is it good for production? Why or why not?

Answer: No, it is not good. The low recall means the model misses most intruders, which is dangerous. High accuracy can be misleading if most images have no intruders. Improving recall is critical even if accuracy drops slightly.

Key Result

For Raspberry Pi deployment, balance inference speed and accuracy; prioritize recall for safety-critical tasks.