Pre-trained detection models in TensorFlow - Model Metrics & Evaluation

Metrics & Evaluation - Pre-trained detection models

Which metric matters for Pre-trained detection models and WHY

For pre-trained detection models, the key metrics are Precision, Recall, and Mean Average Precision (mAP). Precision tells us how many detected objects are actually correct. Recall tells us how many real objects the model found. mAP combines these to show overall detection quality. These metrics matter because detection models must find objects accurately and avoid false alarms.

Confusion matrix for object detection (simplified)

      | Predicted Object | No Object |
      |------------------|-----------|
      | True Object      | TP        | FN        |
      | No Object        | FP        | TN        |

      TP = Correctly detected objects
      FP = False detections (wrong objects)
      FN = Missed objects (not detected)
      TN = Correctly ignored background
    

Note: In detection, TN is often very large and less informative, so focus is on TP, FP, FN.

Precision vs Recall tradeoff with examples

Imagine a security camera detecting people:

• High Precision, Low Recall: The camera only alerts when very sure. Few false alarms, but it may miss some people.
• High Recall, Low Precision: The camera alerts for almost everyone, catching all people but also many false alarms.

Choosing the right balance depends on the use case. For safety, high recall is important to not miss anyone. For convenience, high precision avoids false alerts.

What good vs bad metric values look like

Good detection model metrics:

• Precision and Recall both above 0.8 (80%)
• mAP above 0.5 (50%)

Bad detection model metrics:

• Precision below 0.5 (many false detections)
• Recall below 0.5 (many missed objects)
• mAP below 0.3 (poor overall detection)

Good models find most objects correctly and avoid false alarms.

Common pitfalls in metrics for detection models

• Accuracy paradox: High accuracy can be misleading if most images have no objects.
• Data leakage: Using test images in training inflates metrics falsely.
• Overfitting: Very high training metrics but poor test metrics means model memorizes, not generalizes.
• Ignoring IoU threshold: Detection quality depends on Intersection over Union threshold; metrics vary with this setting.

Self-check question

Your pre-trained detection model has 98% accuracy but only 12% recall on detecting cars. Is it good for production? Why or why not?

Answer: No, it is not good. The high accuracy is misleading because most images may not have cars, so the model guesses "no car" often and is correct. But 12% recall means it misses 88% of cars, which is very poor for detection. The model fails to find most cars, so it is not reliable.

Key Result

Precision, Recall, and mAP are key to evaluate pre-trained detection models, balancing correct detections and missed objects.

Practice

1. What is the main advantage of using a pre-trained detection model in TensorFlow?

easy

A. You can detect objects without training the model yourself

B. It always gives 100% accurate results

C. It requires no input image preprocessing

D. It only works with black and white images

Solution

1. Step 1: Understand pre-trained model purpose

   Pre-trained detection models are trained on large datasets and ready to use for object detection tasks without extra training.

2. Step 2: Identify the main benefit

   This means you save time and resources by not training from scratch and can detect objects directly.

3. Final Answer:

   You can detect objects without training the model yourself -> Option A

4. Quick Check:

   Pre-trained means ready to use without training [OK]

Hint: Pre-trained means no training needed before use [OK]

Common Mistakes:

• Thinking pre-trained models are always perfect
• Ignoring the need for image preprocessing
• Assuming models only work on certain image types

2. Which of the following is the correct way to load a pre-trained detection model from TensorFlow Hub?

easy

A. model = hub.load_model('ssd_mobilenet_v2')

B. model = tf.load_model('ssd_mobilenet_v2')

C. model = hub.load('https://tfhub.dev/tensorflow/ssd_mobilenet_v2/2')

D. model = tfhub.load('ssd_mobilenet_v2')

Solution

1. Step 1: Recall TensorFlow Hub loading syntax

   The correct function to load models from TensorFlow Hub is hub.load() with the full URL or handle.

2. Step 2: Identify the correct option

   model = hub.load('https://tfhub.dev/tensorflow/ssd_mobilenet_v2/2') uses hub.load() with a valid URL, which is the correct method.

3. Final Answer:

   model = hub.load('https://tfhub.dev/tensorflow/ssd_mobilenet_v2/2') -> Option C

4. Quick Check:

   Use hub.load() with URL to load models [OK]

Hint: Use hub.load() with full model URL [OK]

Common Mistakes:

• Using tf.load_model which does not exist
• Confusing hub.load_model with hub.load
• Missing full URL in hub.load

3. Given the following code snippet, what is the shape of the input tensor passed to the detection model?

import tensorflow as tf
image = tf.io.read_file('image.jpg')
image = tf.image.decode_jpeg(image, channels=3)
input_tensor = tf.expand_dims(tf.image.convert_image_dtype(image, tf.float32), 0)
print(input_tensor.shape)

medium

A. (height, width, 3)

B. (1, height, width, 3)

C. (3, height, width)

D. (height, width)

Solution

1. Step 1: Understand image preprocessing steps

   The image is decoded to shape (height, width, 3) for RGB channels, then converted to float32.

2. Step 2: Analyze tf.expand_dims usage

   tf.expand_dims adds a batch dimension at axis 0, making shape (1, height, width, 3).

3. Final Answer:

   (1, height, width, 3) -> Option B

4. Quick Check:

   Batch dimension added first, then image shape [OK]

Hint: expand_dims adds batch size as first dimension [OK]

Common Mistakes:

• Forgetting batch dimension
• Mixing channel order
• Confusing shape with no batch

4. You run a pre-trained detection model but get an error: ValueError: Input tensor must have 4 dimensions. What is the most likely cause?

medium

A. The input tensor is missing the batch dimension

B. The input image was not converted to float32

C. The model URL is incorrect

D. The image file is corrupted

Solution

1. Step 1: Understand model input requirements

   Pre-trained detection models expect input tensors with shape (batch_size, height, width, channels), which means 4 dimensions.

2. Step 2: Identify cause of 4D tensor error

   If the input tensor lacks the batch dimension, it will have only 3 dimensions, causing this error.

3. Final Answer:

   The input tensor is missing the batch dimension -> Option A

4. Quick Check:

   Batch dimension missing causes 4D input error [OK]

Hint: Check if input tensor has batch dimension (4 dims) [OK]

Common Mistakes:

• Assuming float32 conversion fixes dimension error
• Blaming model URL for input shape errors
• Ignoring batch dimension requirement

5. You want to detect objects in multiple images using a pre-trained TensorFlow detection model. Which approach correctly prepares a batch of images for the model?

hard

A. Run the model separately on each image without batching

B. Pass a list of image file paths directly to the model

C. Concatenate images along the channel axis to shape (height, width, batch_size*3)

D. Stack individual preprocessed images into a tensor with shape (batch_size, height, width, 3)

Solution

1. Step 1: Understand batch input format

   Pre-trained detection models expect a 4D tensor with batch size as the first dimension, so multiple images must be stacked into one tensor.

2. Step 2: Evaluate options for batch preparation

   Stack individual preprocessed images into a tensor with shape (batch_size, height, width, 3) correctly stacks images into shape (batch_size, height, width, 3). Options B and C do not produce valid input tensors. Run the model separately on each image without batching works but is inefficient.

3. Final Answer:

   Stack individual preprocessed images into a tensor with shape (batch_size, height, width, 3) -> Option D

4. Quick Check:

   Batch images as 4D tensor for detection [OK]

Hint: Batch images by stacking into 4D tensor [OK]

Common Mistakes:

• Passing file paths instead of tensors
• Concatenating along wrong axis
• Ignoring batch processing benefits