EfficientNet scaling in Computer Vision - ML Experiment: Train & Evaluate

Experiment - EfficientNet scaling

Problem:You want to classify images using EfficientNet, but your current model is too large and overfits the training data.

Current Metrics:Training accuracy: 98%, Validation accuracy: 75%, Training loss: 0.05, Validation loss: 0.85

Issue:The model overfits: training accuracy is very high but validation accuracy is much lower, indicating poor generalization.

Your Task

Reduce overfitting by applying EfficientNet scaling principles to balance model size and accuracy, aiming for validation accuracy >85% with training accuracy <92%.

You can only adjust the EfficientNet model scaling parameters (width, depth, resolution).

Do not change the dataset or training procedure (optimizer, epochs, batch size).

Hint 1

Hint 2

Hint 3

Solution

Computer Vision

import tensorflow as tf
from tensorflow.keras.applications import EfficientNetB0
from tensorflow.keras.layers import Dense, GlobalAveragePooling2D
from tensorflow.keras.models import Model

# Load EfficientNetB0 base model with input shape 224x224x3
base_model = EfficientNetB0(include_top=False, input_shape=(224, 224, 3), weights='imagenet')

# Freeze base model layers to reduce overfitting
base_model.trainable = False

# Add classification head
x = base_model.output
x = GlobalAveragePooling2D()(x)
outputs = Dense(10, activation='softmax')(x)  # Assuming 10 classes
model = Model(inputs=base_model.input, outputs=outputs)

# Compile model
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# Assume X_train, y_train, X_val, y_val are preloaded datasets
# Train with frozen base model
history = model.fit(X_train, y_train, epochs=10, batch_size=32, validation_data=(X_val, y_val))

# Unfreeze some layers for fine-tuning with lower learning rate
base_model.trainable = True
for layer in base_model.layers[:-20]:
    layer.trainable = False

model.compile(optimizer=tf.keras.optimizers.Adam(1e-5), loss='sparse_categorical_crossentropy', metrics=['accuracy'])

history_finetune = model.fit(X_train, y_train, epochs=10, batch_size=32, validation_data=(X_val, y_val))

Reduced input image resolution to 224x224 to balance computation and accuracy.

Used EfficientNetB0 (smallest EfficientNet) instead of larger variants to reduce model size.

Froze base model layers initially to prevent overfitting and trained only classification head.

Fine-tuned last 20 layers with a low learning rate to improve validation accuracy without overfitting.

Results Interpretation

Before: Training accuracy 98%, Validation accuracy 75%, high overfitting.

After: Training accuracy 90%, Validation accuracy 87%, better balance and generalization.

Scaling EfficientNet properly and freezing layers helps reduce overfitting and improves validation accuracy by balancing model complexity and training.

Bonus Experiment

Try increasing input image resolution to 260x260 and use EfficientNetB1 to see if validation accuracy improves further without overfitting.

💡 Hint

Increase resolution and model size carefully; monitor validation loss to avoid overfitting.

Practice

1. What is the main idea behind EfficientNet scaling in computer vision models?

easy

A. It uses only higher image resolution without changing the model.

B. It only increases the number of layers to improve accuracy.

C. It reduces model size by removing layers randomly.

D. It scales depth, width, and resolution together using fixed constants.

Solution

1. Step 1: Understand EfficientNet scaling components

   EfficientNet scales three model dimensions: depth (layers), width (channels), and input resolution together.

2. Step 2: Recognize the use of constants

   It uses constants alpha, beta, gamma with a scaling factor phi to balance these dimensions.

3. Final Answer:

   It scales depth, width, and resolution together using fixed constants. -> Option D

4. Quick Check:

   EfficientNet scales depth, width, resolution together [OK]

Hint: Remember: EfficientNet scales depth, width, and resolution together [OK]

Common Mistakes:

• Thinking it only increases layers
• Assuming it changes only resolution
• Believing it randomly removes layers

2. Which formula correctly represents the compound scaling method used in EfficientNet for depth (d), width (w), and resolution (r)?

easy

A. d = phi * alpha, w = phi * beta, r = phi * gamma

B. d = alpha + phi, w = beta + phi, r = gamma + phi

C. d = alpha^phi, w = beta^phi, r = gamma^phi

D. d = alpha / phi, w = beta / phi, r = gamma / phi

Solution

1. Step 1: Recall EfficientNet scaling formula

   EfficientNet uses exponential scaling: depth = alpha^phi, width = beta^phi, resolution = gamma^phi.

2. Step 2: Compare options with formula

   Only d = alpha^phi, w = beta^phi, r = gamma^phi matches the exponential form with constants raised to the power phi.

3. Final Answer:

   d = alpha^phi, w = beta^phi, r = gamma^phi -> Option C

4. Quick Check:

   Uses exponentiation alpha^phi [OK]

Hint: Look for exponential scaling with phi as power [OK]

Common Mistakes:

• Using multiplication instead of exponentiation
• Adding phi instead of exponentiating
• Dividing constants by phi

3. Given alpha=1.2, beta=1.1, gamma=1.15, and phi=2, what is the scaled depth (d) using EfficientNet scaling?

medium

A. 1.2^2 = 1.44

B. 1.2 * 2 = 2.4

C. 1.2 + 2 = 3.2

D. 2 / 1.2 = 1.67

Solution

1. Step 1: Apply the formula for depth scaling

   Depth d = alpha^phi = 1.2^2 = 1.44.

2. Step 2: Calculate the value

   1.2 squared equals 1.44, matching 1.2^2 = 1.44.

3. Final Answer:

   1.44 -> Option A

4. Quick Check:

   1.2^2 = 1.44 [OK]

Hint: Raise alpha to the power phi for depth [OK]

Common Mistakes:

• Multiplying alpha by phi instead of exponentiating
• Adding phi to alpha
• Dividing phi by alpha

4. Identify the error in this Python code snippet for EfficientNet scaling:

alpha, beta, gamma, phi = 1.2, 1.1, 1.15, 2
depth = alpha * phi
width = beta ** phi
resolution = gamma ** phi

medium

A. Depth should be alpha ** phi, not alpha * phi

B. Width should be beta * phi, not beta ** phi

C. Resolution should be gamma * phi, not gamma ** phi

D. No error, the code is correct

Solution

1. Step 1: Review EfficientNet scaling formula

   Depth should be scaled as alpha raised to phi (alpha ** phi), not multiplied.

2. Step 2: Check code for depth calculation

   Code uses alpha * phi which is incorrect; width and resolution use exponentiation correctly.

3. Final Answer:

   Depth should be alpha ** phi, not alpha * phi -> Option A

4. Quick Check:

   Depth uses exponentiation (**), not multiplication (*) [OK]

Hint: Depth uses exponentiation, not multiplication [OK]

Common Mistakes:

• Confusing multiplication with exponentiation
• Assuming width or resolution calculations are wrong
• Thinking code has no errors

5. You want to scale an EfficientNet model with phi=3, alpha=1.2, beta=1.1, gamma=1.15. Which of these sets of scaled values (depth, width, resolution) is closest to the correct scaling?

hard

A. (1.2+3, 1.1+3, 1.15+3) = (4.2, 4.1, 4.15)

B. (1.2^3, 1.1^3, 1.15^3) ≈ (1.73, 1.33, 1.52)

C. (3*1.2, 3*1.1, 3*1.15) = (3.6, 3.3, 3.45)

D. (3/1.2, 3/1.1, 3/1.15) ≈ (2.5, 2.73, 2.61)

Solution

1. Step 1: Apply compound scaling formula

   Scale each dimension by raising constants to the power phi: depth = 1.2^3, width = 1.1^3, resolution = 1.15^3.

2. Step 2: Calculate approximate values

   1.2^3 ≈ 1.73, 1.1^3 ≈ 1.33, 1.15^3 ≈ 1.52, matching (1.2^3, 1.1^3, 1.15^3) ≈ (1.73, 1.33, 1.52).

3. Final Answer:

   (1.73, 1.33, 1.52) -> Option B

4. Quick Check:

   1.2^3 ≈ 1.73, 1.1^3 ≈ 1.33, 1.15^3 ≈ 1.52 [OK]

Hint: Use powers, not multiplication or addition for scaling [OK]

Common Mistakes:

• Multiplying constants by phi instead of exponentiating
• Adding phi to constants
• Dividing phi by constants