Mobile deployment (TFLite, Core ML) in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Mobile deployment (TFLite, Core ML)

Which metric matters for Mobile deployment (TFLite, Core ML) and WHY

When deploying computer vision models on mobile devices using TFLite or Core ML, the key metrics to focus on are model size, inference speed, and accuracy.

Model size matters because mobile devices have limited storage and memory. Smaller models load faster and use less space.

Inference speed is important because users expect quick responses. A slow model leads to poor user experience.

Accuracy remains critical to ensure the model makes correct predictions despite being smaller or faster.

Balancing these metrics ensures the model works well on mobile without draining battery or causing delays.

Confusion matrix example for a mobile image classifier

      Actual \ Predicted | Cat | Dog | Bird
      -------------------------------------
      Cat                | 45  | 3   | 2
      Dog                | 4   | 40  | 6
      Bird               | 1   | 5   | 44
    

This matrix shows how many images of each animal were correctly or incorrectly classified. It helps measure accuracy, precision, and recall for each class.

Precision vs Recall tradeoff in mobile deployment

Imagine a mobile app that detects if a photo contains a dog.

• High precision means when the app says "dog," it is usually right. This avoids false alarms but might miss some dogs.
• High recall means the app finds most dogs in photos but might sometimes say "dog" when there isn't one.

On mobile, you might prefer high precision to avoid annoying users with wrong alerts, even if some dogs are missed.

But if missing a dog is critical (like a safety app), you might choose high recall.

What good vs bad metrics look like for mobile deployment

• Good: Model size under 10MB, inference time under 100ms, accuracy above 85%
• Bad: Model size over 50MB (slow to load), inference time over 1 second (laggy), accuracy below 70% (many wrong predictions)

Good metrics mean the app feels fast, uses little space, and predicts well. Bad metrics cause slow, clunky apps with poor results.

Common pitfalls in mobile deployment metrics

• Ignoring latency: A model with high accuracy but slow inference is bad for mobile.
• Overfitting: A model that works great on test data but poorly on real mobile images.
• Data leakage: Training data too similar to test data inflates accuracy falsely.
• Ignoring battery use: Complex models drain battery quickly.
• Not testing on real devices: Metrics from desktop may not reflect mobile performance.

Self-check question

Your mobile model has 98% accuracy but takes 2 seconds to classify one image. Is it good for production? Why or why not?

Answer: No, it is not good. Even with high accuracy, 2 seconds per image is too slow for a smooth mobile experience. Users expect quick results, so inference speed must improve.

Key Result

For mobile deployment, balancing small model size, fast inference, and good accuracy is key to a good user experience.