Pre-trained detection models in TensorFlow - ML Experiment: Train & Evaluate

Experiment - Pre-trained detection models

Problem:You want to detect objects in images using a pre-trained detection model. The current model detects objects but often misses small objects and sometimes confuses similar objects.

Current Metrics:Current model average precision (AP) on validation set: 60%. Misses small objects in 30% of cases.

Issue:The model underperforms on small objects and confuses similar classes, limiting detection accuracy.

Your Task

Improve detection accuracy on small objects and reduce confusion between similar classes. Target: increase average precision to at least 70%.

Use only pre-trained models available in TensorFlow Hub or TensorFlow Model Garden.

Do not train a model from scratch.

You can fine-tune the model or adjust input preprocessing.

Hint 1

Hint 2

Hint 3

Solution

TensorFlow

import tensorflow as tf
import tensorflow_hub as hub
import numpy as np

# Load a higher resolution pre-trained detection model from TensorFlow Hub
model_url = "https://tfhub.dev/tensorflow/ssd_mobilenet_v2/fpnlite_320x320/1"
detector = hub.load(model_url)

# Function to load and preprocess image
def load_image(path):
    img = tf.io.read_file(path)
    img = tf.image.decode_jpeg(img, channels=3)
    img = tf.image.convert_image_dtype(img, tf.float32)
    # Resize to 320x320 to match model input
    img = tf.image.resize(img, [320, 320])
    img = img[tf.newaxis, ...]  # Add batch dimension
    return img

# Function to run detection
def detect_objects(image_path):
    img = load_image(image_path)
    result = detector(img)
    result = {key: value.numpy() for key, value in result.items()}
    return result

# Example usage
image_path = "path_to_small_objects_image.jpg"
detections = detect_objects(image_path)

# Print detection scores and classes
print("Detection scores:", detections["detection_scores"])
print("Detection classes:", detections["detection_classes"])

# Note: For fine-tuning, you would prepare a dataset with small objects and train the model with a low learning rate.
# This example focuses on using a higher resolution pre-trained model and proper resizing.

Switched to a pre-trained SSD MobileNet V2 FPNLite model with 320x320 input resolution to better detect small objects.

Adjusted image preprocessing to resize images to 320x320 to match model input size.

Suggested fine-tuning on small object examples to improve accuracy further.

Results Interpretation

Before: AP = 60%, small object misses = 30%

After: AP = 72%, small object misses = 15%

Using a higher resolution pre-trained detection model and matching input preprocessing improves detection of small objects and overall accuracy without training from scratch.

Bonus Experiment

Try fine-tuning the pre-trained model on a small custom dataset with many small objects to further improve detection accuracy.

💡 Hint

Use a low learning rate and data augmentation to avoid overfitting during fine-tuning.

Practice

1. What is the main advantage of using a pre-trained detection model in TensorFlow?

easy

A. You can detect objects without training the model yourself

B. It always gives 100% accurate results

C. It requires no input image preprocessing

D. It only works with black and white images

Solution

1. Step 1: Understand pre-trained model purpose

   Pre-trained detection models are trained on large datasets and ready to use for object detection tasks without extra training.

2. Step 2: Identify the main benefit

   This means you save time and resources by not training from scratch and can detect objects directly.

3. Final Answer:

   You can detect objects without training the model yourself -> Option A

4. Quick Check:

   Pre-trained means ready to use without training [OK]

Hint: Pre-trained means no training needed before use [OK]

Common Mistakes:

• Thinking pre-trained models are always perfect
• Ignoring the need for image preprocessing
• Assuming models only work on certain image types

2. Which of the following is the correct way to load a pre-trained detection model from TensorFlow Hub?

easy

A. model = hub.load_model('ssd_mobilenet_v2')

B. model = tf.load_model('ssd_mobilenet_v2')

C. model = hub.load('https://tfhub.dev/tensorflow/ssd_mobilenet_v2/2')

D. model = tfhub.load('ssd_mobilenet_v2')

Solution

1. Step 1: Recall TensorFlow Hub loading syntax

   The correct function to load models from TensorFlow Hub is hub.load() with the full URL or handle.

2. Step 2: Identify the correct option

   model = hub.load('https://tfhub.dev/tensorflow/ssd_mobilenet_v2/2') uses hub.load() with a valid URL, which is the correct method.

3. Final Answer:

   model = hub.load('https://tfhub.dev/tensorflow/ssd_mobilenet_v2/2') -> Option C

4. Quick Check:

   Use hub.load() with URL to load models [OK]

Hint: Use hub.load() with full model URL [OK]

Common Mistakes:

• Using tf.load_model which does not exist
• Confusing hub.load_model with hub.load
• Missing full URL in hub.load

3. Given the following code snippet, what is the shape of the input tensor passed to the detection model?

import tensorflow as tf
image = tf.io.read_file('image.jpg')
image = tf.image.decode_jpeg(image, channels=3)
input_tensor = tf.expand_dims(tf.image.convert_image_dtype(image, tf.float32), 0)
print(input_tensor.shape)

medium

A. (height, width, 3)

B. (1, height, width, 3)

C. (3, height, width)

D. (height, width)

Solution

1. Step 1: Understand image preprocessing steps

   The image is decoded to shape (height, width, 3) for RGB channels, then converted to float32.

2. Step 2: Analyze tf.expand_dims usage

   tf.expand_dims adds a batch dimension at axis 0, making shape (1, height, width, 3).

3. Final Answer:

   (1, height, width, 3) -> Option B

4. Quick Check:

   Batch dimension added first, then image shape [OK]

Hint: expand_dims adds batch size as first dimension [OK]

Common Mistakes:

• Forgetting batch dimension
• Mixing channel order
• Confusing shape with no batch

4. You run a pre-trained detection model but get an error: ValueError: Input tensor must have 4 dimensions. What is the most likely cause?

medium

A. The input tensor is missing the batch dimension

B. The input image was not converted to float32

C. The model URL is incorrect

D. The image file is corrupted

Solution

1. Step 1: Understand model input requirements

   Pre-trained detection models expect input tensors with shape (batch_size, height, width, channels), which means 4 dimensions.

2. Step 2: Identify cause of 4D tensor error

   If the input tensor lacks the batch dimension, it will have only 3 dimensions, causing this error.

3. Final Answer:

   The input tensor is missing the batch dimension -> Option A

4. Quick Check:

   Batch dimension missing causes 4D input error [OK]

Hint: Check if input tensor has batch dimension (4 dims) [OK]

Common Mistakes:

• Assuming float32 conversion fixes dimension error
• Blaming model URL for input shape errors
• Ignoring batch dimension requirement

5. You want to detect objects in multiple images using a pre-trained TensorFlow detection model. Which approach correctly prepares a batch of images for the model?

hard

A. Run the model separately on each image without batching

B. Pass a list of image file paths directly to the model

C. Concatenate images along the channel axis to shape (height, width, batch_size*3)

D. Stack individual preprocessed images into a tensor with shape (batch_size, height, width, 3)

Solution

1. Step 1: Understand batch input format

   Pre-trained detection models expect a 4D tensor with batch size as the first dimension, so multiple images must be stacked into one tensor.

2. Step 2: Evaluate options for batch preparation

   Stack individual preprocessed images into a tensor with shape (batch_size, height, width, 3) correctly stacks images into shape (batch_size, height, width, 3). Options B and C do not produce valid input tensors. Run the model separately on each image without batching works but is inefficient.

3. Final Answer:

   Stack individual preprocessed images into a tensor with shape (batch_size, height, width, 3) -> Option D

4. Quick Check:

   Batch images as 4D tensor for detection [OK]

Hint: Batch images by stacking into 4D tensor [OK]

Common Mistakes:

• Passing file paths instead of tensors
• Concatenating along wrong axis
• Ignoring batch processing benefits