Raspberry Pi deployment in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Raspberry Pi deployment

Which metric matters for Raspberry Pi deployment and WHY

When deploying computer vision models on a Raspberry Pi, the key metrics to watch are inference speed and accuracy. Inference speed tells us how fast the model can make predictions on the device, which is important because Raspberry Pi has limited computing power. Accuracy shows how well the model recognizes images or objects. We want a balance: a model fast enough to run smoothly on the Pi but still accurate enough to be useful.

Confusion matrix example for Raspberry Pi model

Suppose the model detects cats and dogs. Here is a confusion matrix from test data:

      | Predicted Cat | Predicted Dog |
      |--------------|---------------|
      | True Cat: 40 | False Dog: 5  |
      | False Cat: 3 | True Dog: 52  |
    

Total samples = 40 + 5 + 3 + 52 = 100

From this, we calculate:

• Precision for Cat = TP / (TP + FP) = 40 / (40 + 3) = 0.93
• Recall for Cat = TP / (TP + FN) = 40 / (40 + 5) = 0.89

Precision vs Recall tradeoff with Raspberry Pi examples

Imagine a home security camera on Raspberry Pi detecting intruders:

• High Precision: Few false alarms. The camera rarely mistakes a pet for an intruder. This avoids annoying alerts.
• High Recall: The camera catches almost every real intruder. Missing one is risky.

On Raspberry Pi, if the model is too complex to run fast, you might lower recall to keep speed. Or if safety is critical, you accept slower speed for higher recall.

Good vs Bad metric values for Raspberry Pi deployment

Good: Accuracy above 85%, inference time under 1 second per image, precision and recall balanced around 0.9. This means the model is both fast and reliable.

Bad: Accuracy below 70%, inference time over 3 seconds, or very low recall (below 0.5). This means the model is either too slow or misses too many objects, making it unusable on Raspberry Pi.

Common pitfalls in Raspberry Pi deployment metrics

• Accuracy paradox: High accuracy but poor recall if data is unbalanced (e.g., mostly background images).
• Data leakage: Testing on images very similar to training can give false high accuracy.
• Overfitting: Model works well on test data but slow or inaccurate on real Raspberry Pi images.
• Ignoring latency: A model with great accuracy but too slow to run on Raspberry Pi is not practical.

Self-check question

Your Raspberry Pi model has 98% accuracy but only 12% recall on detecting intruders. Is it good for production? Why or why not?

Answer: No, it is not good. The low recall means the model misses most intruders, which is dangerous. High accuracy can be misleading if most images have no intruders. Improving recall is critical even if accuracy drops slightly.

Key Result

For Raspberry Pi deployment, balance inference speed and accuracy; prioritize recall for safety-critical tasks.

Practice

1. What is the main advantage of deploying a machine learning model on a Raspberry Pi?

easy

A. It allows running ML models locally without internet connection

B. It increases the training speed of the model

C. It automatically improves model accuracy

D. It requires no power to operate

Solution

1. Step 1: Understand Raspberry Pi deployment context

   Raspberry Pi is a small device that can run ML models locally, meaning it does not need to send data to the cloud.

2. Step 2: Identify the main benefit

   Running models locally allows offline use and faster response without internet dependency.

3. Final Answer:

   It allows running ML models locally without internet connection -> Option A

4. Quick Check:

   Local inference = no internet needed [OK]

Hint: Local means no internet needed for predictions [OK]

Common Mistakes:

• Confusing deployment with training speed
• Thinking deployment improves accuracy automatically
• Assuming Raspberry Pi needs no power

2. Which Python package is commonly used to run TensorFlow Lite models on a Raspberry Pi?

easy

A. tensorflow

B. tflite_runtime

C. scikit-learn

D. opencv-python

Solution

1. Step 1: Identify the package for TensorFlow Lite on Raspberry Pi

   The lightweight package designed for running TFLite models on small devices is tflite_runtime.

2. Step 2: Differentiate from other packages

   tensorflow is large and not optimized for Pi; scikit-learn is for classical ML; opencv-python is for image processing.

3. Final Answer:

   tflite_runtime -> Option B

4. Quick Check:

   TFLite on Pi = tflite_runtime [OK]

Hint: Use tflite_runtime for lightweight TensorFlow Lite on Pi [OK]

Common Mistakes:

• Using full tensorflow package on Raspberry Pi
• Confusing scikit-learn with TensorFlow Lite
• Thinking OpenCV runs ML models directly

3. Given this code snippet on Raspberry Pi, what will output_data contain?

import numpy as np
from tflite_runtime.interpreter import Interpreter

interpreter = Interpreter(model_path='model.tflite')
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

input_shape = input_details[0]['shape']
input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)
interpreter.set_tensor(input_details[0]['index'], input_data)
interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])

medium

A. The model's prediction output as a numpy array

B. The input data shape as a tuple

C. A syntax error due to missing import

D. An empty list because invoke() was not called

Solution

1. Step 1: Understand the TFLite interpreter flow

   The code loads a TFLite model, prepares input data, sets it, runs inference with invoke(), then gets output tensor.

2. Step 2: Identify what output_data holds

   After invoke(), get_tensor() returns the model's prediction output as a numpy array.

3. Final Answer:

   The model's prediction output as a numpy array -> Option A

4. Quick Check:

   invoke() then get_tensor() = model output [OK]

Hint: invoke() runs model; get_tensor() fetches predictions [OK]

Common Mistakes:

• Thinking output_data is input shape
• Forgetting to call invoke() before get_tensor()
• Assuming output_data is empty or error

4. You run this Raspberry Pi TFLite code but get an error: ValueError: Cannot set tensor: Dimension mismatch. What is the likely cause?

input_shape = interpreter.get_input_details()[0]['shape']
input_data = np.array([1, 2, 3], dtype=np.float32)
interpreter.set_tensor(interpreter.get_input_details()[0]['index'], input_data)

medium

A. The data type of input_data is wrong

B. The model file path is incorrect

C. The interpreter was not allocated tensors

D. Input data shape does not match model's expected input shape

Solution

1. Step 1: Check input data shape vs model input shape

   The model expects input shape from input_shape, but input_data is a 1D array of length 3, likely mismatched.

2. Step 2: Understand error cause

   Setting tensor with wrong shape causes dimension mismatch error.

3. Final Answer:

   Input data shape does not match model's expected input shape -> Option D

4. Quick Check:

   Shape mismatch = ValueError on set_tensor [OK]

Hint: Match input_data shape exactly to model input shape [OK]

Common Mistakes:

• Ignoring shape mismatch and changing file path
• Forgetting to call allocate_tensors()
• Assuming data type causes dimension error

5. You want to deploy a computer vision model on Raspberry Pi to detect objects in real-time video. Which approach best balances speed and accuracy?

hard

A. Send video frames to a cloud server for processing and get results back

B. Use a large TensorFlow model and run it directly on Raspberry Pi CPU

C. Convert the model to TensorFlow Lite and use quantization for faster inference

D. Use OpenCV only without any ML model for detection

Solution

1. Step 1: Consider Raspberry Pi hardware limits

   Raspberry Pi has limited CPU and memory, so large models run slowly.

2. Step 2: Choose model optimization for speed and accuracy

   Converting to TensorFlow Lite and applying quantization reduces model size and speeds up inference with minimal accuracy loss.

3. Step 3: Evaluate other options

   Sending to cloud adds latency; OpenCV alone lacks ML detection power; large models are too slow locally.

4. Final Answer:

   Convert the model to TensorFlow Lite and use quantization for faster inference -> Option C

5. Quick Check:

   TFLite + quantization = fast, accurate on Pi [OK]

Hint: Optimize model with TFLite quantization for Pi speed [OK]

Common Mistakes:

• Trying to run large models without optimization
• Relying on cloud adds delay and needs internet
• Using only OpenCV misses ML detection benefits