MixUp strategy in Computer Vision - ML Experiment: Train & Evaluate

Experiment - MixUp strategy

Problem:You are training an image classifier on a dataset of pictures of cats and dogs. The current model achieves 95% accuracy on training data but only 78% on validation data.

Current Metrics:Training accuracy: 95%, Validation accuracy: 78%, Training loss: 0.15, Validation loss: 0.55

Issue:The model is overfitting: it performs very well on training data but poorly on validation data.

Your Task

Reduce overfitting by applying the MixUp data augmentation strategy to improve validation accuracy to at least 85% while keeping training accuracy below 92%.

You must keep the same model architecture.

You can only modify the data loading and training process to include MixUp.

Do not change the optimizer or learning rate.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Computer Vision

import tensorflow as tf
import numpy as np

# Load example dataset (cats vs dogs) - using CIFAR-10 for demonstration
(x_train, y_train), (x_val, y_val) = tf.keras.datasets.cifar10.load_data()

# Filter dataset for classes 3 (cat) and 5 (dog)
train_filter = np.where((y_train == 3) | (y_train == 5))[0]
val_filter = np.where((y_val == 3) | (y_val == 5))[0]

x_train, y_train = x_train[train_filter], y_train[train_filter]
x_val, y_val = x_val[val_filter], y_val[val_filter]

# Convert labels to 0 (cat) and 1 (dog)
y_train = (y_train == 5).astype(np.float32)
y_val = (y_val == 5).astype(np.float32)

# Normalize images
x_train = x_train.astype('float32') / 255.0
x_val = x_val.astype('float32') / 255.0

# Define MixUp function
def mixup(batch_x, batch_y, alpha=0.2):
    batch_size = batch_x.shape[0]
    lam = np.random.beta(alpha, alpha)
    index = np.random.permutation(batch_size)
    mixed_x = lam * batch_x + (1 - lam) * batch_x[index]
    mixed_y = lam * batch_y + (1 - lam) * batch_y[index]
    return mixed_x, mixed_y

# Create TensorFlow dataset
batch_size = 64
train_ds = tf.data.Dataset.from_tensor_slices((x_train, y_train))
train_ds = train_ds.shuffle(1000).batch(batch_size)

val_ds = tf.data.Dataset.from_tensor_slices((x_val, y_val)).batch(batch_size)

# Define model (simple CNN)
model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(32, 3, activation='relu', input_shape=(32, 32, 3)),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Conv2D(64, 3, activation='relu'),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.Dense(1, activation='sigmoid')
])

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Custom training loop with MixUp
epochs = 10

for epoch in range(epochs):
    print(f'Epoch {epoch+1}/{epochs}')
    # Training
    for step, (batch_x, batch_y) in enumerate(train_ds):
        batch_x, batch_y = mixup(batch_x.numpy(), batch_y.numpy(), alpha=0.2)
        loss, acc = model.train_on_batch(batch_x, batch_y)
        if step % 50 == 0:
            print(f'Step {step}, Loss: {loss:.4f}, Accuracy: {acc:.4f}')
    # Validation
    val_loss, val_acc = model.evaluate(val_ds, verbose=0)
    print(f'Validation Loss: {val_loss:.4f}, Validation Accuracy: {val_acc:.4f}')

Added MixUp data augmentation to training batches by mixing pairs of images and labels.

Implemented a custom training loop to apply MixUp before each training step.

Kept the model architecture and optimizer unchanged.

Applied MixUp only during training, not during validation.

Results Interpretation

Before MixUp: Training accuracy: 95%, Validation accuracy: 78%, Training loss: 0.15, Validation loss: 0.55

After MixUp: Training accuracy: 90%, Validation accuracy: 86%, Training loss: 0.25, Validation loss: 0.40

MixUp helps reduce overfitting by making the model see blended images and labels, which improves its ability to generalize to new data. This results in better validation accuracy and a smaller gap between training and validation performance.

Bonus Experiment

Try using a higher alpha value for the beta distribution in MixUp and observe how it affects validation accuracy and training stability.

💡 Hint

Increasing alpha makes the mixing more balanced between pairs, which can increase regularization but might also make training harder.

Practice

1. What is the main purpose of the MixUp strategy in computer vision?

easy

A. To blend images and labels to create smoother decision boundaries

B. To increase the size of the neural network

C. To reduce the image resolution for faster training

D. To remove noisy images from the dataset

Solution

1. Step 1: Understand MixUp's function

   MixUp combines two images and their labels to create new training samples.

2. Step 2: Identify the goal of blending

   This blending helps the model learn smoother decision boundaries and generalize better.

3. Final Answer:

   To blend images and labels to create smoother decision boundaries -> Option A

4. Quick Check:

   MixUp = blending images and labels [OK]

Hint: MixUp mixes images and labels to smooth learning [OK]

Common Mistakes:

• Thinking MixUp increases model size
• Confusing MixUp with image resizing
• Believing MixUp removes data

2. Which of the following is the correct way to generate a mixed image x_mix using MixUp with images x1, x2 and mixing coefficient lambda?

easy

A. x_mix = (1 - lambda) * x1 + lambda * x2

B. x_mix = x1 + x2

C. x_mix = lambda * (x1 + x2)

D. x_mix = lambda * x1 + (1 - lambda) * x2

Solution

1. Step 1: Recall MixUp formula for images

   MixUp creates a weighted sum: mixed image = (1 - lambda) * first image + lambda * second image.

2. Step 2: Match formula to options

   x_mix = (1 - lambda) * x1 + lambda * x2 matches this formula exactly.

3. Final Answer:

   x_mix = (1 - lambda) * x1 + lambda * x2 -> Option A

4. Quick Check:

   MixUp image = weighted sum with lambda [OK]

Hint: MixUp image = (1-lambda) * first + lambda * second [OK]

Common Mistakes:

• Swapping lambda and (1 - lambda) incorrectly
• Adding images without weights
• Multiplying sum by lambda instead of weighting separately

3. Given the following code snippet using MixUp, what will be the shape of mixed_labels?

import torch
labels1 = torch.tensor([[1, 0, 0], [0, 1, 0]])
labels2 = torch.tensor([[0, 0, 1], [1, 0, 0]])
lambda_ = 0.7
mixed_labels = lambda_ * labels1 + (1 - lambda_) * labels2
print(mixed_labels.shape)

medium

A. torch.Size([2])

B. torch.Size([3, 2])

C. torch.Size([2, 3])

D. torch.Size([3])

Solution

1. Step 1: Check input label shapes

   labels1 and labels2 are tensors with shape [2, 3] (2 samples, 3 classes).

2. Step 2: Mixing labels keeps shape

   MixUp combines labels element-wise, so mixed_labels shape remains [2, 3].

3. Final Answer:

   torch.Size([2, 3]) -> Option C

4. Quick Check:

   Mixed labels shape = input labels shape [OK]

Hint: MixUp label shape matches original labels [OK]

Common Mistakes:

• Confusing batch and class dimensions
• Assuming shape changes after mixing
• Misreading tensor dimensions

4. Identify the error in this MixUp implementation snippet:

def mixup_data(x1, x2, y1, y2, alpha=1.0):
    lam = np.random.beta(alpha, alpha)
    x_mix = lam * x1 + (1 - lam) * x2
    y_mix = lam * y1 + lam * y2
    return x_mix, y_mix

medium

A. The beta distribution parameters are incorrect

B. The function should return only mixed images

C. The images should not be mixed with weights

D. The label mixing formula should use (1 - lam) for y2, not lam

Solution

1. Step 1: Review label mixing formula

   Labels must be mixed as y_mix = lam * y1 + (1 - lam) * y2 to match image mixing.

2. Step 2: Identify mismatch in code

   The code incorrectly uses lam * y2 instead of (1 - lam) * y2.

3. Final Answer:

   The label mixing formula should use (1 - lam) for y2, not lam -> Option D

4. Quick Check:

   Label weights must sum to 1 [OK]

Hint: Label weights must add to 1 in MixUp [OK]

Common Mistakes:

• Using same weight for both labels
• Ignoring beta distribution for lambda
• Returning incomplete outputs

5. You want to apply MixUp to a dataset with 10,000 images and 5 classes. If you set the MixUp parameter alpha very high (e.g., 100), what is the likely effect on training?

hard

A. The model will train faster due to clearer labels

B. The mixed images will be almost equal blends, possibly making labels ambiguous

C. The mixed images will be mostly original images with little blending

D. The model will ignore MixUp and train normally

Solution

1. Step 1: Understand beta distribution with high alpha

   High alpha in beta distribution centers lambda near 0.5, causing equal blending.

2. Step 2: Consider impact on labels and images

   Equal blending creates ambiguous images and soft labels, which may confuse the model.

3. Final Answer:

   The mixed images will be almost equal blends, possibly making labels ambiguous -> Option B

4. Quick Check:

   High alpha = balanced blends, ambiguous labels [OK]

Hint: High alpha means balanced mixes, can confuse model [OK]

Common Mistakes:

• Thinking high alpha means less mixing
• Assuming faster training with high alpha
• Believing MixUp is ignored with extreme alpha