CV project workflow in Computer Vision - ML Experiment: Train & Evaluate

Experiment - CV project workflow

Problem:You want to build a computer vision model to classify images into categories. Currently, you have a simple model trained on a small dataset.

Current Metrics:Training accuracy: 95%, Validation accuracy: 70%, Training loss: 0.15, Validation loss: 0.60

Issue:The model is overfitting. It performs very well on training data but poorly on validation data.

Your Task

Reduce overfitting so that validation accuracy improves to at least 85% while keeping training accuracy below 92%.

You can only modify the model architecture and training process.

You cannot add more data or use external datasets.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Computer Vision

import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Data preparation with augmentation
train_datagen = ImageDataGenerator(
    rescale=1./255,
    rotation_range=20,
    width_shift_range=0.2,
    height_shift_range=0.2,
    horizontal_flip=True,
    validation_split=0.2
)

val_datagen = ImageDataGenerator(
    rescale=1./255,
    validation_split=0.2
)

train_generator = train_datagen.flow_from_directory(
    'data/train',
    target_size=(64, 64),
    batch_size=32,
    class_mode='categorical',
    subset='training'
)

validation_generator = val_datagen.flow_from_directory(
    'data/train',
    target_size=(64, 64),
    batch_size=32,
    class_mode='categorical',
    subset='validation'
)

# Model with dropout to reduce overfitting
model = models.Sequential([
    layers.Conv2D(32, (3,3), activation='relu', input_shape=(64,64,3)),
    layers.MaxPooling2D(2,2),
    layers.Dropout(0.25),
    layers.Conv2D(64, (3,3), activation='relu'),
    layers.MaxPooling2D(2,2),
    layers.Dropout(0.25),
    layers.Flatten(),
    layers.Dense(128, activation='relu'),
    layers.Dropout(0.5),
    layers.Dense(train_generator.num_classes, activation='softmax')
])

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
              loss='categorical_crossentropy',
              metrics=['accuracy'])

# Early stopping callback
early_stop = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=5, restore_best_weights=True)

history = model.fit(
    train_generator,
    epochs=50,
    validation_data=validation_generator,
    callbacks=[early_stop]
)

# Output training and validation accuracy
train_acc = history.history['accuracy'][-1] * 100
val_acc = history.history['val_accuracy'][-1] * 100
train_loss = history.history['loss'][-1]
val_loss = history.history['val_loss'][-1]

print(f'Training accuracy: {train_acc:.2f}%')
print(f'Validation accuracy: {val_acc:.2f}%')
print(f'Training loss: {train_loss:.4f}')
print(f'Validation loss: {val_loss:.4f}')

Added dropout layers after convolution and dense layers to reduce overfitting.

Applied data augmentation to increase data variety during training.

Used early stopping to stop training when validation loss stops improving.

Kept learning rate moderate and batch size at 32 for stable training.

Results Interpretation

Before: Training accuracy 95%, Validation accuracy 70%, Training loss 0.15, Validation loss 0.60

After: Training accuracy 90%, Validation accuracy 87%, Training loss 0.25, Validation loss 0.35

Adding dropout and data augmentation helps the model generalize better, reducing overfitting and improving validation accuracy.

Bonus Experiment

Try using transfer learning with a pre-trained model like MobileNetV2 to improve accuracy further.

💡 Hint

Freeze the base layers and train only the top layers first, then fine-tune some base layers.

Practice

1. Which step comes first in a typical computer vision project workflow?

easy

A. Monitor model performance

B. Deploy the model to production

C. Tune hyperparameters

D. Define the problem and collect data

Solution

1. Step 1: Understand the project start

   The first step is to clearly define what problem you want to solve and gather the images or videos needed.

2. Step 2: Recognize the order of workflow steps

   Data collection must happen before training, tuning, or deployment.

3. Final Answer:

   Define the problem and collect data -> Option D

4. Quick Check:

   First step = Define problem and collect data [OK]

Hint: Start with problem definition and data collection [OK]

Common Mistakes:

• Thinking deployment is the first step
• Skipping problem definition
• Ignoring data collection importance

2. Which of the following is the correct syntax to split data into training and testing sets in Python using scikit-learn?

easy

A. train_test_split(data, test_size=0.2)

B. split_train_test(data, 0.2)

C. train_test(data, test=0.2)

D. train_test_split(data, test=0.2)

Solution

1. Step 1: Recall scikit-learn function name and parameters

   The correct function is train_test_split with parameter test_size to specify test data fraction.

2. Step 2: Check parameter correctness

   train_test_split(data, test_size=0.2) uses correct function and parameter names.

3. Final Answer:

   train_test_split(data, test_size=0.2) -> Option A

4. Quick Check:

   Correct function and parameter = train_test_split(data, test_size=0.2) [OK]

Hint: Remember exact function and parameter names from scikit-learn [OK]

Common Mistakes:

• Using wrong function name
• Using incorrect parameter names
• Confusing test_size with test

3. Given this code snippet for training a simple CNN model, what will be the printed output after training for 1 epoch?

import tensorflow as tf
model = tf.keras.Sequential([
  tf.keras.layers.Conv2D(16, 3, activation='relu', input_shape=(28,28,1)),
  tf.keras.layers.Flatten(),
  tf.keras.layers.Dense(10, activation='softmax')
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
history = model.fit(x_train, y_train, epochs=1, batch_size=32)
print(history.history['accuracy'][0])

medium

A. An error because 'accuracy' is not in history

B. The loss value after training

C. A float value between 0 and 1 representing training accuracy

D. The number of training samples

Solution

1. Step 1: Understand model.fit output

   The history object stores metrics per epoch. Accessing history.history['accuracy'][0] gives training accuracy after first epoch.

2. Step 2: Confirm metric requested

   Since metrics=['accuracy'] was set, accuracy is recorded and printed as a float between 0 and 1.

3. Final Answer:

   A float value between 0 and 1 representing training accuracy -> Option C

4. Quick Check:

   history.history['accuracy'][0] = training accuracy [OK]

Hint: history.history['accuracy'][0] holds first epoch accuracy [OK]

Common Mistakes:

• Confusing accuracy with loss
• Expecting an error accessing accuracy
• Thinking it prints sample count

4. You trained a model but it performs poorly on new images. Which step in the workflow might be causing this issue?

medium

A. Monitoring was set up correctly

B. Data preparation was insufficient or incorrect

C. Hyperparameters were tuned perfectly

D. Model deployment was done too early

Solution

1. Step 1: Analyze poor model performance cause

   Poor results on new data often mean the model did not learn well, usually due to bad or insufficient data preparation.

2. Step 2: Eliminate unrelated options

   Deployment timing, perfect hyperparameters, or monitoring setup do not directly cause poor initial performance.

3. Final Answer:

   Data preparation was insufficient or incorrect -> Option B

4. Quick Check:

   Poor performance = bad data prep [OK]

Hint: Check data prep first when model fails on new data [OK]

Common Mistakes:

• Blaming deployment timing
• Assuming hyperparameters are always perfect
• Ignoring data quality issues

5. In a computer vision project, after deploying your model, you notice accuracy drops over time. What is the best next step to maintain model performance?

hard

A. Collect new data and retrain the model regularly

B. Stop monitoring since model is deployed

C. Reduce the size of the training dataset

D. Ignore the drop as normal and do nothing

Solution

1. Step 1: Understand model drift after deployment

   Models can lose accuracy as data changes. Collecting new data and retraining helps adapt to changes.

2. Step 2: Evaluate other options

   Stopping monitoring or ignoring drops will worsen performance. Reducing training data size is counterproductive.

3. Final Answer:

   Collect new data and retrain the model regularly -> Option A

4. Quick Check:

   Maintain performance = retrain with new data [OK]

Hint: Retrain model regularly with fresh data after deployment [OK]

Common Mistakes:

• Ignoring monitoring after deployment
• Reducing training data size
• Assuming model never needs updates