TensorFlow Object Detection API

Introduction

The TensorFlow Object Detection API helps computers find and label objects in pictures or videos easily.

You want to detect cars and people in traffic camera images.

You need to count items on a store shelf automatically.

You want to find faces in photos for a photo app.

You want to track animals in wildlife videos.

You want to build a security system that detects intruders.

Syntax

TensorFlow

import tensorflow as tf
from object_detection.utils import config_util
from object_detection.builders import model_builder

# Load pipeline config and build model
configs = config_util.get_configs_from_pipeline_file('pipeline.config')
model_config = configs['model']
detection_model = model_builder.build(model_config=model_config, is_training=False)

# Load checkpoint
ckpt = tf.compat.v2.train.Checkpoint(model=detection_model)
ckpt.restore('checkpoint_path').expect_partial()

# Run detection on an image
@tf.function
def detect_fn(image):
    image, shapes = detection_model.preprocess(image)
    prediction_dict = detection_model.predict(image, shapes)
    detections = detection_model.postprocess(prediction_dict, shapes)
    return detections

This code shows how to load a pre-trained detection model and run it on images.

You need a pipeline config file and checkpoint files from a trained model.

Examples

This example shows how to run detection on an image and draw boxes around detected objects.

TensorFlow

from object_detection.utils import visualization_utils as viz_utils
import numpy as np

# Assume image_np is a numpy array of the image
input_tensor = tf.convert_to_tensor(np.expand_dims(image_np, 0), dtype=tf.float32)
detections = detect_fn(input_tensor)

# Visualize results
viz_utils.visualize_boxes_and_labels_on_image_array(
    image_np,
    detections['detection_boxes'][0].numpy(),
    detections['detection_classes'][0].numpy().astype(np.int32),
    detections['detection_scores'][0].numpy(),
    category_index,
    use_normalized_coordinates=True,
    line_thickness=8)

# Now image_np has boxes drawn on it

This shows how to start training a model using the API's training script.

TensorFlow

# To train a model, use the command line:
# python model_main_tf2.py --pipeline_config_path=pipeline.config --model_dir=training/ --alsologtostderr

Sample Model

This program loads a pre-trained TensorFlow object detection model, runs it on a test image, and prints the top 3 detected objects with their class IDs and confidence scores.

TensorFlow

import tensorflow as tf
import numpy as np
from object_detection.utils import config_util
from object_detection.builders import model_builder
from object_detection.utils import visualization_utils as viz_utils
import cv2

# Load pipeline config and build model
configs = config_util.get_configs_from_pipeline_file('pipeline.config')
model_config = configs['model']
detection_model = model_builder.build(model_config=model_config, is_training=False)

# Restore checkpoint
ckpt = tf.compat.v2.train.Checkpoint(model=detection_model)
ckpt.restore('checkpoint/checkpoint-0').expect_partial()

@tf.function
def detect_fn(image):
    image, shapes = detection_model.preprocess(image)
    prediction_dict = detection_model.predict(image, shapes)
    detections = detection_model.postprocess(prediction_dict, shapes)
    return detections

# Load image
image_path = 'test_image.jpg'
image_np = cv2.imread(image_path)
image_np = cv2.cvtColor(image_np, cv2.COLOR_BGR2RGB)
input_tensor = tf.convert_to_tensor(np.expand_dims(image_np, 0), dtype=tf.uint8)

# Run detection
output_dict = detect_fn(input_tensor)

# Extract detection info
num_detections = int(output_dict['num_detections'][0])
detection_classes = output_dict['detection_classes'][0].numpy().astype(np.int32)
detection_scores = output_dict['detection_scores'][0].numpy()
detection_boxes = output_dict['detection_boxes'][0].numpy()

# Print top 3 detections
for i in range(3):
    print(f"Object {i+1}: Class {detection_classes[i]}, Score {detection_scores[i]:.2f}")

Important Notes

You need to download or train a model and have the pipeline.config and checkpoint files ready.

Class IDs correspond to labels in your label map file.

Images must be converted to RGB and expanded in batch dimension before detection.

Summary

The TensorFlow Object Detection API helps find objects in images or videos.

You load a model, prepare images, run detection, and get boxes, classes, and scores.

It works well for many real-world tasks like security, retail, and wildlife monitoring.

Practice

1. What is the main purpose of the TensorFlow Object Detection API?

easy

A. To translate languages

B. To generate text from images

C. To find and locate objects in images or videos

D. To create 3D models from photos

Solution

1. Step 1: Understand the API's function

   The TensorFlow Object Detection API is designed to detect objects by finding their locations in images or videos.

2. Step 2: Compare options with the API's purpose

   Only To find and locate objects in images or videos matches this purpose, while others describe unrelated tasks.

3. Final Answer:

   To find and locate objects in images or videos -> Option C

4. Quick Check:

   Object detection = find objects [OK]

Hint: Remember: Object Detection means locating objects [OK]

Common Mistakes:

• Confusing object detection with image translation
• Thinking it generates text from images
• Mixing it up with 3D modeling

2. Which of the following is the correct way to load a pre-trained object detection model in TensorFlow Object Detection API?

easy

A. model = tf.saved_model.load('path_to_model')

B. model = tf.keras.models.load_model('path_to_model')

C. model = tf.load_model('path_to_model')

D. model = tf.object_detection.load('path_to_model')

Solution

1. Step 1: Identify the correct TensorFlow function for loading saved models

   TensorFlow uses tf.saved_model.load() to load saved models, including object detection models.

2. Step 2: Check other options for correctness

   tf.keras.models.load_model() loads Keras models, not specifically object detection saved models; tf.load_model() and tf.object_detection.load() do not exist.

3. Final Answer:

   model = tf.saved_model.load('path_to_model') -> Option A

4. Quick Check:

   Use tf.saved_model.load() for saved models [OK]

Hint: Use tf.saved_model.load() to load detection models [OK]

Common Mistakes:

• Using tf.keras.models.load_model() for detection models
• Trying non-existent functions like tf.load_model()
• Confusing model loading with training functions

3. Given the following code snippet, what will be the output type of detection_boxes after running detection on an image?

import tensorflow as tf
model = tf.saved_model.load('ssd_mobilenet_v2')
image = tf.constant(image_np)
input_tensor = tf.expand_dims(image, 0)
output_dict = model(input_tensor)
detection_boxes = output_dict['detection_boxes']

medium

A. A tensor of shape [1, num_detections, 4] with box coordinates

B. A list of bounding box coordinates as tuples

C. A numpy array of class labels

D. A scalar float value representing confidence

Solution

1. Step 1: Understand the output of TensorFlow Object Detection model

   The model returns a dictionary with keys like 'detection_boxes', which is a tensor containing bounding box coordinates for detected objects.

2. Step 2: Check the shape and type of detection_boxes

   It is a tensor with shape [1, num_detections, 4], where 4 represents box coordinates (ymin, xmin, ymax, xmax).

3. Final Answer:

   A tensor of shape [1, num_detections, 4] with box coordinates -> Option A

4. Quick Check:

   detection_boxes = tensor with box coords [OK]

Hint: Detection boxes are tensors with shape [1, N, 4] [OK]

Common Mistakes:

• Thinking detection_boxes is a list or numpy array
• Confusing boxes with class labels or scores
• Expecting a scalar instead of a tensor

4. You run the following code but get a KeyError: 'detection_scores'. What is the most likely cause?

output_dict = model(input_tensor)
scores = output_dict['detection_scores']

medium

A. The image is not preprocessed

B. The input_tensor shape is incorrect

C. The model was not loaded properly

D. The model output dictionary keys are bytes, not strings

Solution

1. Step 1: Understand the KeyError cause

   A KeyError means the key 'detection_scores' is not found in output_dict.

2. Step 2: Check common output format issue

   TensorFlow Object Detection API sometimes returns keys as bytes (b'detection_scores'), so using string keys causes KeyError.

3. Final Answer:

   The model output dictionary keys are bytes, not strings -> Option D

4. Quick Check:

   Use bytes keys or decode keys to strings [OK]

Hint: Check if output keys are bytes, not strings [OK]

Common Mistakes:

• Assuming input shape error causes KeyError
• Thinking model loading failed without error
• Ignoring key type mismatch in output dictionary

5. You want to improve detection accuracy on a custom dataset using TensorFlow Object Detection API. Which approach is best?

hard

A. Use the model without any training on your data

B. Fine-tune a pre-trained model on your dataset

C. Train a model from scratch with random weights

D. Only increase the number of detection classes without training

Solution

1. Step 1: Understand transfer learning benefits

   Fine-tuning a pre-trained model uses learned features and adapts them to your data, improving accuracy efficiently.

2. Step 2: Evaluate other options

   Training from scratch requires much data and time; using a model without training won't adapt to new data; increasing classes without training won't improve accuracy.

3. Final Answer:

   Fine-tune a pre-trained model on your dataset -> Option B

4. Quick Check:

   Fine-tuning = best for custom data accuracy [OK]

Hint: Fine-tune pre-trained models for better custom results [OK]

Common Mistakes:

• Training from scratch without enough data
• Using pre-trained model without adaptation
• Changing classes without retraining