YOLO concept overview in TensorFlow - ML Experiment: Train & Evaluate

Experiment - YOLO concept overview

Problem:You want to detect objects in images quickly and accurately using the YOLO (You Only Look Once) model.

Current Metrics:Training accuracy: 95%, Validation accuracy: 70%, Training loss: 0.15, Validation loss: 0.45

Issue:The model is overfitting: training accuracy is high but validation accuracy is much lower, showing poor generalization.

Your Task

Reduce overfitting so that validation accuracy improves to at least 85% while keeping training accuracy below 92%.

You can only change model architecture and training hyperparameters.

You cannot change the dataset or input image size.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

TensorFlow

import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Define a simple YOLO-like model architecture
inputs = tf.keras.Input(shape=(224, 224, 3))
x = layers.Conv2D(16, (3,3), activation='relu', padding='same')(inputs)
x = layers.MaxPooling2D(2,2)(x)
x = layers.Conv2D(32, (3,3), activation='relu', padding='same')(x)
x = layers.MaxPooling2D(2,2)(x)
x = layers.Dropout(0.3)(x)  # Added dropout
x = layers.Conv2D(64, (3,3), activation='relu', padding='same')(x)
x = layers.MaxPooling2D(2,2)(x)
x = layers.Dropout(0.3)(x)  # Added dropout
x = layers.Flatten()(x)
x = layers.Dense(128, activation='relu')(x)
x = layers.Dropout(0.4)(x)  # Added dropout
outputs = layers.Dense(7*7*30, activation='sigmoid')(x)  # Example output for YOLO grid

model = models.Model(inputs=inputs, outputs=outputs)

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0005),
              loss='binary_crossentropy',
              metrics=['accuracy'])

# Data augmentation setup
train_datagen = ImageDataGenerator(
    rescale=1./255,
    rotation_range=15,
    width_shift_range=0.1,
    height_shift_range=0.1,
    horizontal_flip=True,
    validation_split=0.2
)

# Assuming train_dir is the directory with training images and labels
train_generator = train_datagen.flow_from_directory(
    'train_dir',
    target_size=(224, 224),
    batch_size=32,
    class_mode='categorical',
    subset='training'
)

validation_generator = train_datagen.flow_from_directory(
    'train_dir',
    target_size=(224, 224),
    batch_size=32,
    class_mode='categorical',
    subset='validation'
)

# Early stopping callback
early_stop = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=5, restore_best_weights=True)

history = model.fit(
    train_generator,
    epochs=50,
    validation_data=validation_generator,
    callbacks=[early_stop]
)

Added dropout layers after convolutional and dense layers to reduce overfitting.

Implemented data augmentation to increase training data variety.

Reduced learning rate from default to 0.0005 for smoother training.

Added early stopping to stop training when validation loss stops improving.

Results Interpretation

Before: Training accuracy 95%, Validation accuracy 70%, Training loss 0.15, Validation loss 0.45

After: Training accuracy 90%, Validation accuracy 87%, Training loss 0.25, Validation loss 0.30

Adding dropout and data augmentation helps the model generalize better, reducing overfitting and improving validation accuracy.

Bonus Experiment

Try using a pretrained backbone like MobileNet or ResNet as the feature extractor in YOLO to improve accuracy further.

💡 Hint

Use transfer learning by loading pretrained weights and fine-tuning only the last layers.

Practice

1. What is the main idea behind the YOLO (You Only Look Once) model in object detection?

easy

A. It splits the image into a grid and predicts bounding boxes and classes for each cell.

B. It processes each object separately in multiple passes.

C. It uses a sliding window to scan the image multiple times.

D. It detects objects by comparing image patches with a database.

Solution

1. Step 1: Understand YOLO's approach to image processing

   YOLO divides the input image into a grid and predicts bounding boxes and class probabilities for each grid cell in one pass.

2. Step 2: Compare with other detection methods

   Unlike sliding window or multiple passes, YOLO predicts all objects at once, making it fast and efficient.

3. Final Answer:

   It splits the image into a grid and predicts bounding boxes and classes for each cell. -> Option A

4. Quick Check:

   YOLO grid prediction = It splits the image into a grid and predicts bounding boxes and classes for each cell. [OK]

Hint: YOLO predicts all objects in one grid pass [OK]

Common Mistakes:

• Thinking YOLO scans image multiple times
• Confusing YOLO with sliding window methods
• Believing YOLO processes objects one by one

2. Which of the following is the correct way to describe YOLO's output format?

easy

A. A single large tensor containing bounding box coordinates and class probabilities for all grid cells.

B. A list of detected objects with coordinates and classes, one by one.

C. Multiple small tensors each representing a detected object separately.

D. A heatmap showing object presence without bounding boxes.

Solution

1. Step 1: Recall YOLO's output structure

   YOLO outputs a single large tensor that encodes bounding boxes and class probabilities for every grid cell simultaneously.

2. Step 2: Eliminate other options

   Options A and C describe separate outputs per object, which is not how YOLO works. A heatmap showing object presence without bounding boxes describes a heatmap, which is not YOLO's output.

3. Final Answer:

   A single large tensor containing bounding box coordinates and class probabilities for all grid cells. -> Option A

4. Quick Check:

   YOLO outputs one big tensor = A single large tensor containing bounding box coordinates and class probabilities for all grid cells. [OK]

Hint: YOLO outputs one big tensor for all predictions [OK]

Common Mistakes:

• Thinking YOLO outputs separate tensors per object
• Confusing YOLO output with heatmaps
• Assuming YOLO outputs a list instead of a tensor

3. Given a YOLO model that splits an image into a 7x7 grid and predicts 2 bounding boxes per cell with 20 classes, what is the shape of the output tensor?

medium

A. (7, 7, 28)

B. (7, 7, 50)

C. (7, 7, 44)

D. (7, 7, 30)

Solution

1. Step 1: Calculate output features per grid cell

   Each cell predicts 2 bounding boxes. Each box has 5 values (x, y, width, height, confidence), so 2*5=10. Plus 20 class probabilities, total 10+20=30 per cell.

2. Step 2: Calculate total output tensor shape

   Grid size is 7x7, so output shape is (7, 7, 30). But (7, 7, 50) says (7,7,50), so check carefully.

3. Step 3: Re-examine calculations

   Actually, each bounding box has 5 values, so 2 boxes = 10 values. Adding 20 class probabilities gives 30 per cell, so output shape is (7,7,30).

4. Final Answer:

   (7, 7, 30) -> Option D

5. Quick Check:

   Output shape = grid size * (2*5 + 20) = (7,7,30) [OK]

Hint: Output depth = 2*5 + number_of_classes [OK]

Common Mistakes:

• Adding bounding boxes and classes incorrectly
• Confusing number of boxes with classes
• Miscounting bounding box parameters

4. You wrote a YOLO model that outputs a tensor of shape (7, 7, 25) but your dataset has 3 bounding boxes per cell and 20 classes. What is the likely error?

medium

A. Number of classes should be 5, not 20.

B. Output tensor depth is too small; should be 3*5 + 20 = 35, not 25.

C. Output tensor shape is correct; no error.

D. Grid size should be 5x5, not 7x7.

Solution

1. Step 1: Calculate expected output depth

   Each bounding box has 5 values, so 3 boxes = 15 values. Adding 20 classes gives 15 + 20 = 35 values per grid cell.

2. Step 2: Compare with given output shape

   The model outputs (7, 7, 25), which is less than expected 35, indicating the output tensor depth is too small.

3. Final Answer:

   Output tensor depth is too small; should be 3*5 + 20 = 35, not 25. -> Option B

4. Quick Check:

   Output depth mismatch = Output tensor depth is too small; should be 3*5 + 20 = 35, not 25. [OK]

Hint: Check output depth = boxes*5 + classes [OK]

Common Mistakes:

• Ignoring bounding box parameters count
• Assuming output shape is always (7,7,25)
• Mixing grid size with output depth

5. You want to improve YOLO's real-time detection speed on a mobile device. Which approach is best while keeping reasonable accuracy?

hard

A. Reduce the grid size to 3x3 and keep 2 bounding boxes per cell.

B. Increase the number of bounding boxes per cell to 5 for better coverage.

C. Use a smaller backbone network and reduce input image size.

D. Increase input image size to get more details.

Solution

1. Step 1: Understand speed vs accuracy trade-offs

   Reducing model size and input resolution speeds up inference but may reduce accuracy slightly.

2. Step 2: Evaluate options for mobile optimization

   Reduce the grid size to 3x3 and keep 2 bounding boxes per cell reduces grid size too much, hurting detection detail. Increase the number of bounding boxes per cell to 5 for better coverage increases complexity, slowing down. Increase input image size to get more details increases input size, slowing down.

3. Step 3: Choose best option

   Using a smaller backbone and smaller input size balances speed and accuracy well for mobile devices.

4. Final Answer:

   Use a smaller backbone network and reduce input image size. -> Option C

5. Quick Check:

   Smaller model + input = faster detection [OK]

Hint: Smaller model and input size boost speed [OK]

Common Mistakes:

• Reducing grid size too much loses detail
• Increasing bounding boxes slows model
• Increasing input size hurts speed