Cutout and CutMix in Computer Vision

Introduction

Cutout and CutMix help models learn better by changing images during training. They make the model stronger and less likely to make mistakes.

When training image classifiers to improve accuracy.

When you want your model to handle missing or occluded parts in images.

When you have limited data and want to create more variety.

When you want to reduce overfitting by making training harder.

When you want to improve model robustness to real-world image changes.

Syntax

Computer Vision

Cutout(image, mask_size)
CutMix(image1, label1, image2, label2, alpha)

Cutout removes a square patch from the image by masking it.

CutMix mixes two images by cutting a patch from one and pasting it on the other, mixing their labels too.

Examples

This removes a 50x50 pixel square from the original image.

Computer Vision

cutout_image = Cutout(original_image, mask_size=50)

This mixes two images and their labels using a random patch size controlled by alpha.

Computer Vision

mixed_image, mixed_label = CutMix(image1, label1, image2, label2, alpha=1.0)

Sample Model

This code shows how to apply Cutout and CutMix on simple dummy images. Cutout masks a square area in the white image. CutMix mixes a patch from a black image into the white image and combines their labels accordingly.

Computer Vision

import numpy as np
import matplotlib.pyplot as plt

def cutout(image, mask_size):
    h, w, _ = image.shape
    y = np.random.randint(h)
    x = np.random.randint(w)
    y1 = np.clip(y - mask_size // 2, 0, h)
    y2 = np.clip(y + mask_size // 2, 0, h)
    x1 = np.clip(x - mask_size // 2, 0, w)
    x2 = np.clip(x + mask_size // 2, 0, w)
    image_copy = image.copy()
    image_copy[y1:y2, x1:x2, :] = 0
    return image_copy

def cutmix(image1, label1, image2, label2, alpha=1.0):
    lam = np.random.beta(alpha, alpha)
    h, w, _ = image1.shape
    cut_rat = np.sqrt(1. - lam)
    cut_w = int(w * cut_rat)
    cut_h = int(h * cut_rat)

    cx = np.random.randint(w)
    cy = np.random.randint(h)

    x1 = np.clip(cx - cut_w // 2, 0, w)
    y1 = np.clip(cy - cut_h // 2, 0, h)
    x2 = np.clip(cx + cut_w // 2, 0, w)
    y2 = np.clip(cy + cut_h // 2, 0, h)

    new_image = image1.copy()
    new_image[y1:y2, x1:x2, :] = image2[y1:y2, x1:x2, :]

    lam_adjusted = 1 - ((x2 - x1) * (y2 - y1) / (w * h))
    new_label = lam_adjusted * label1 + (1 - lam_adjusted) * label2

    return new_image, new_label

# Create dummy images and labels
image1 = np.ones((100, 100, 3), dtype=np.uint8) * 255  # white image
image2 = np.zeros((100, 100, 3), dtype=np.uint8)         # black image
label1 = 1.0  # class 1
label2 = 0.0  # class 0

# Apply Cutout
cutout_img = cutout(image1, mask_size=30)

# Apply CutMix
mixed_img, mixed_label = cutmix(image1, label1, image2, label2, alpha=1.0)

print(f"CutMix mixed label: {mixed_label:.2f}")

Important Notes

Cutout helps the model learn to recognize objects even if parts are missing.

CutMix mixes images and labels, so the model learns from combined features.

Both methods increase data variety without needing more images.

Summary

Cutout removes a patch from an image to improve model robustness.

CutMix combines two images and their labels to create new training samples.

Both help reduce overfitting and improve accuracy in image tasks.

Practice

1. What is the main purpose of the Cutout technique in computer vision?

easy

A. To remove a patch from an image to improve model robustness

B. To combine two images by overlaying one on the other

C. To increase image resolution for better detail

D. To convert images to grayscale for simpler processing

Solution

1. Step 1: Understand Cutout's function

   Cutout removes a small patch from an image during training to force the model to learn from other parts.

2. Step 2: Compare options

   Only To remove a patch from an image to improve model robustness describes removing a patch to improve robustness. Others describe different techniques.

3. Final Answer:

   To remove a patch from an image to improve model robustness -> Option A

4. Quick Check:

   Cutout = patch removal [OK]

Hint: Cutout = cut a patch out of the image [OK]

Common Mistakes:

• Confusing Cutout with CutMix
• Thinking Cutout increases resolution
• Assuming Cutout converts image color

2. Which of the following is the correct way to mix labels in CutMix augmentation in Python using PyTorch?

easy

A. new_label = label1 * lambda + label2 * (1 - lambda)

B. new_label = label1 + label2

C. new_label = label1 - label2

D. new_label = label1 / label2

Solution

1. Step 1: Recall CutMix formula

   CutMix creates a new label by mixing two labels weighted by lambda and (1 - lambda).

2. Step 2: Match formula to options

   new_label = label1 * lambda + label2 * (1 - lambda) correctly shows weighted sum of two labels. Others are incorrect operations.

3. Final Answer:

   new_label = label1 * lambda + label2 * (1 - lambda) -> Option A

4. Quick Check:

   CutMix = weighted sum [OK]

Hint: CutMix blends labels weighted by lambda [OK]

Common Mistakes:

• Using addition without weights
• Subtracting or dividing labels incorrectly
• Confusing CutMix with Cutout

3. Given the following code snippet applying Cutout, what will be the shape of the output image tensor?

import torch
import torchvision.transforms as T

cutout = T.RandomErasing(p=1.0, scale=(0.1, 0.1))
image = torch.randn(3, 224, 224)
output = cutout(image)
print(output.shape)

medium

A. torch.Size([3, 112, 112])

B. torch.Size([1, 224, 224])

C. torch.Size([224, 224, 3])

D. torch.Size([3, 224, 224])

Solution

1. Step 1: Understand RandomErasing effect

   RandomErasing (Cutout) masks part of the image but does not change its shape.

2. Step 2: Check input and output shapes

   Input is (3, 224, 224), output remains same shape after Cutout.

3. Final Answer:

   torch.Size([3, 224, 224]) -> Option D

4. Quick Check:

   Cutout keeps shape same [OK]

Hint: Cutout masks but keeps image size [OK]

Common Mistakes:

• Thinking Cutout changes image dimensions
• Confusing channel order with shape
• Assuming output is batch dimension added

4. The following code attempts to apply CutMix but causes an error. What is the main issue?

def cutmix(image1, image2, lam):
    bbx1, bby1, bbx2, bby2 = rand_bbox(image1.size(), lam)
    image1[:, bbx1:bbx2, bby1:bby2] = image2[:, bbx1:bbx2, bby1:bby2]
    return image1

# image1 and image2 are tensors of shape (3, 224, 448)
# lam is a float between 0 and 1

medium

A. rand_bbox returns coordinates in (x, y) but slicing expects (height, width)

B. rand_bbox expects size as tuple (H, W) instead of tensor.size() (C, H, W)

C. The function modifies image1 in-place which is not allowed

D. The function does not return the mixed label

Solution

1. Step 1: Analyze rand_bbox input and slicing

   rand_bbox expects size as (height, width), but image1.size() returns (channels, height, width).

2. Step 2: Identify mismatch in size argument

   Passing image1.size() (C, H, W) instead of (H, W) causes rand_bbox to compute wrong coordinates, leading to slicing errors.

3. Final Answer:

   rand_bbox expects size as tuple (H, W) instead of tensor.size() (C, H, W) -> Option B

4. Quick Check:

   Size argument mismatch = error [OK]

Hint: Pass (H, W) size to rand_bbox, not (C, H, W) [OK]

Common Mistakes:

• Ignoring coordinate order difference
• Assuming in-place modification causes error
• Forgetting to return mixed label

5. You want to improve your image classifier's robustness by combining Cutout and CutMix. Which approach correctly applies both augmentations during training?

hard

A. Apply Cutout to each image separately, then combine them with CutMix

B. Apply Cutout only after training with CutMix is complete

C. Apply CutMix first to combine images, then apply Cutout to mask patches on the mixed image

D. Apply CutMix only on validation data and Cutout only on training data

Solution

1. Step 1: Understand augmentation order

   CutMix combines two images, then Cutout can mask parts of the combined image to increase robustness.

2. Step 2: Evaluate options for correct sequence

   Apply CutMix first to combine images, then apply Cutout to mask patches on the mixed image applies CutMix first, then Cutout, which is the correct order to augment the mixed image effectively.

3. Final Answer:

   Apply CutMix first to combine images, then apply Cutout to mask patches on the mixed image -> Option C

4. Quick Check:

   CutMix then Cutout = best practice [OK]

Hint: Mix images first, then mask patches [OK]

Common Mistakes:

• Applying Cutout before mixing images
• Using augmentations only on validation data
• Applying augmentations after training