Cutout and CutMix in Computer Vision

Introduction

Cutout and CutMix help models learn better by changing images during training. They make the model stronger and less likely to make mistakes.

When training image classifiers to improve accuracy.

When you want your model to handle missing or occluded parts in images.

When you have limited data and want to create more variety.

When you want to reduce overfitting by making training harder.

When you want to improve model robustness to real-world image changes.

Syntax

Computer Vision

Cutout(image, mask_size)
CutMix(image1, label1, image2, label2, alpha)

Cutout removes a square patch from the image by masking it.

CutMix mixes two images by cutting a patch from one and pasting it on the other, mixing their labels too.

Examples

This removes a 50x50 pixel square from the original image.

Computer Vision

cutout_image = Cutout(original_image, mask_size=50)

This mixes two images and their labels using a random patch size controlled by alpha.

Computer Vision

mixed_image, mixed_label = CutMix(image1, label1, image2, label2, alpha=1.0)

Sample Model

This code shows how to apply Cutout and CutMix on simple dummy images. Cutout masks a square area in the white image. CutMix mixes a patch from a black image into the white image and combines their labels accordingly.

Computer Vision

import numpy as np
import matplotlib.pyplot as plt

def cutout(image, mask_size):
    h, w, _ = image.shape
    y = np.random.randint(h)
    x = np.random.randint(w)
    y1 = np.clip(y - mask_size // 2, 0, h)
    y2 = np.clip(y + mask_size // 2, 0, h)
    x1 = np.clip(x - mask_size // 2, 0, w)
    x2 = np.clip(x + mask_size // 2, 0, w)
    image_copy = image.copy()
    image_copy[y1:y2, x1:x2, :] = 0
    return image_copy

def cutmix(image1, label1, image2, label2, alpha=1.0):
    lam = np.random.beta(alpha, alpha)
    h, w, _ = image1.shape
    cut_rat = np.sqrt(1. - lam)
    cut_w = int(w * cut_rat)
    cut_h = int(h * cut_rat)

    cx = np.random.randint(w)
    cy = np.random.randint(h)

    x1 = np.clip(cx - cut_w // 2, 0, w)
    y1 = np.clip(cy - cut_h // 2, 0, h)
    x2 = np.clip(cx + cut_w // 2, 0, w)
    y2 = np.clip(cy + cut_h // 2, 0, h)

    new_image = image1.copy()
    new_image[y1:y2, x1:x2, :] = image2[y1:y2, x1:x2, :]

    lam_adjusted = 1 - ((x2 - x1) * (y2 - y1) / (w * h))
    new_label = lam_adjusted * label1 + (1 - lam_adjusted) * label2

    return new_image, new_label

# Create dummy images and labels
image1 = np.ones((100, 100, 3), dtype=np.uint8) * 255  # white image
image2 = np.zeros((100, 100, 3), dtype=np.uint8)         # black image
label1 = 1.0  # class 1
label2 = 0.0  # class 0

# Apply Cutout
cutout_img = cutout(image1, mask_size=30)

# Apply CutMix
mixed_img, mixed_label = cutmix(image1, label1, image2, label2, alpha=1.0)

print(f"CutMix mixed label: {mixed_label:.2f}")

Important Notes

Cutout helps the model learn to recognize objects even if parts are missing.

CutMix mixes images and labels, so the model learns from combined features.

Both methods increase data variety without needing more images.

Summary

Cutout removes a patch from an image to improve model robustness.

CutMix combines two images and their labels to create new training samples.

Both help reduce overfitting and improve accuracy in image tasks.