TensorFlow Lite for mobile - Model Metrics & Evaluation

Metrics & Evaluation - TensorFlow Lite for mobile

Which metric matters for TensorFlow Lite on mobile and WHY

When using TensorFlow Lite on mobile devices, the key metrics to focus on are model size, inference latency, and accuracy. Model size matters because mobile devices have limited storage. Inference latency is important since users expect fast responses. Accuracy remains critical to ensure the model predictions are useful and reliable. Balancing these metrics helps deliver a smooth and effective mobile AI experience.

Confusion matrix example for a mobile image classifier

      | Predicted Cat | Predicted Dog |
      |---------------|---------------|
      | True Cat: 40  | False Dog: 5  |
      | False Cat: 3  | True Dog: 52  |

      Total samples = 40 + 5 + 3 + 52 = 100

      Precision (Cat) = TP / (TP + FP) = 40 / (40 + 3) = 0.93
      Recall (Cat) = TP / (TP + FN) = 40 / (40 + 5) = 0.89
    

This confusion matrix shows how well the model running on mobile predicts cats and dogs. It helps calculate precision and recall to understand prediction quality.

Precision vs Recall tradeoff on mobile

Imagine a mobile app that detects if a photo contains a dog. If the app has high precision, it means when it says "dog," it is usually right. This avoids annoying users with wrong alerts. High recall means the app finds most dog photos, so it rarely misses one. On mobile, you want a balance: too many false alarms (low precision) frustrate users, but missing dogs (low recall) reduces usefulness. Optimizing this tradeoff depends on the app's goal and user experience.

Good vs Bad metric values for TensorFlow Lite models

• Good: Model size under 10MB, inference latency under 100ms, accuracy above 85%
• Bad: Model size over 50MB (too large for mobile), latency over 500ms (slow response), accuracy below 60% (unreliable predictions)

Good metrics mean the model fits mobile limits and gives fast, accurate results. Bad metrics lead to slow apps, wasted storage, and poor user trust.

Common pitfalls in evaluating TensorFlow Lite models

• Ignoring latency: A model with great accuracy but slow inference frustrates mobile users.
• Overfitting: Model performs well on test data but poorly on real mobile inputs.
• Data leakage: Training data accidentally includes test samples, inflating accuracy.
• Ignoring model size: Large models may not run efficiently or fit on devices.

Self-check question

Your TensorFlow Lite model on mobile has 98% accuracy but only 12% recall on detecting rare objects. Is it good for production? Why or why not?

Answer: No, it is not good. While 98% accuracy sounds high, the very low recall means the model misses most rare objects. For rare object detection, recall is critical to catch as many true cases as possible. This model would fail users who need reliable detection.

Key Result

For TensorFlow Lite on mobile, balancing model size, inference speed, and accuracy ensures good user experience and reliable predictions.

Practice

1. What is the main purpose of TensorFlow Lite in mobile applications?

easy

A. To run machine learning models efficiently on mobile devices

B. To train large machine learning models on phones

C. To replace mobile operating systems with AI

D. To connect mobile apps to cloud servers for AI

Solution

1. Step 1: Understand TensorFlow Lite's role

   TensorFlow Lite is designed to run pre-trained ML models on small devices like phones efficiently.

2. Step 2: Differentiate from training or cloud use

   It does not train models on phones or replace OS or require cloud connection for inference.

3. Final Answer:

   To run machine learning models efficiently on mobile devices -> Option A

4. Quick Check:

   TensorFlow Lite = Efficient mobile ML inference [OK]

Hint: Remember: TensorFlow Lite runs models, not trains them [OK]

Common Mistakes:

• Confusing inference with training on mobile
• Thinking it replaces mobile OS
• Assuming it needs cloud connection

2. Which code snippet correctly loads a TensorFlow Lite model in Python?

easy

A. interpreter = tf.lite.Interpreter(model_path='model.tflite')

B. interpreter = tf.load_model('model.tflite')

C. interpreter = tf.tflite.load('model.tflite')

D. interpreter = tf.model.Interpreter('model.tflite')

Solution

1. Step 1: Recall TensorFlow Lite loading syntax

   The correct way is to use tf.lite.Interpreter with model_path argument.

2. Step 2: Check each option's correctness

   Only interpreter = tf.lite.Interpreter(model_path='model.tflite') uses the correct class and argument. Others use invalid methods or classes.

3. Final Answer:

   interpreter = tf.lite.Interpreter(model_path='model.tflite') -> Option A

4. Quick Check:

   Use tf.lite.Interpreter(model_path=...) [OK]

Hint: Use tf.lite.Interpreter with model_path to load models [OK]

Common Mistakes:

• Using non-existent tf.load_model function
• Wrong class names like tf.model.Interpreter
• Missing model_path argument

3. What will be the output of this TensorFlow Lite inference code snippet?

import numpy as np
import tensorflow as tf

interpreter = tf.lite.Interpreter(model_path='model.tflite')
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

input_data = np.array([[1.0, 2.0, 3.0]], dtype=np.float32)
interpreter.set_tensor(input_details[0]['index'], input_data)
interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])
print(output_data)

medium

A. None, because interpreter.invoke() was not called

B. A numpy array with the model's prediction for input [1.0, 2.0, 3.0]

C. An error because input_data shape is wrong

D. A string describing the model architecture

Solution

1. Step 1: Analyze the inference steps

   The code loads the model, allocates tensors, sets input, runs inference, and gets output.

2. Step 2: Confirm output type

   Output is a numpy array with prediction values for the input data provided.

3. Final Answer:

   A numpy array with the model's prediction for input [1.0, 2.0, 3.0] -> Option B

4. Quick Check:

   Output after invoke() = prediction array [OK]

Hint: invoke() runs model; get_tensor() returns prediction array [OK]

Common Mistakes:

• Forgetting to call allocate_tensors() before inference
• Not calling interpreter.invoke() before getting output
• Assuming output is a string or error

4. Identify the error in this TensorFlow Lite Python code snippet:

interpreter = tf.lite.Interpreter(model_path='model.tflite')
input_details = interpreter.get_input_details()
interpreter.allocate_tensors()
input_data = np.array([[1, 2, 3]], dtype=np.float32)
interpreter.set_tensor(input_details[0]['index'], input_data)
interpreter.invoke()

medium

A. input_data should be a list, not a numpy array

B. interpreter.invoke() must be called before set_tensor()

C. allocate_tensors() should be called before get_input_details()

D. model_path argument is missing

Solution

1. Step 1: Check method call order

   allocate_tensors() must be called before get_input_details() to initialize tensors.

2. Step 2: Verify other parts

   input_data as numpy array is correct; invoke() is correctly after set_tensor(); model_path is provided.

3. Final Answer:

   allocate_tensors() should be called before get_input_details() -> Option C

4. Quick Check:

   Call allocate_tensors() before input details [OK]

Hint: Always allocate tensors before querying input/output details [OK]

Common Mistakes:

• Calling get_input_details() before allocate_tensors()
• Using list instead of numpy array for input
• Calling invoke() too early

5. You want to optimize a TensorFlow model for mobile by reducing its size and latency. Which TensorFlow Lite feature should you use?

hard

A. Training the model on mobile devices

B. Increasing the number of layers in the model

C. Using a larger batch size during inference

D. Post-training quantization to convert weights to 8-bit integers

Solution

1. Step 1: Understand optimization goals

   Reducing model size and latency means making it smaller and faster on mobile.

2. Step 2: Identify TensorFlow Lite optimization methods

   Post-training quantization converts weights to 8-bit integers, reducing size and speeding up inference.

3. Step 3: Evaluate other options

   Increasing layers or batch size increases size/latency; training on mobile is not practical for optimization.

4. Final Answer:

   Post-training quantization to convert weights to 8-bit integers -> Option D

5. Quick Check:

   Quantization = smaller, faster model [OK]

Hint: Use quantization to shrink and speed up models [OK]

Common Mistakes:

• Thinking bigger models run faster
• Using large batch sizes on mobile
• Trying to train models on phones