Why architecture choices affect scalability in Prompt Engineering / GenAI - Experiment to Prove It

Experiment - Why architecture choices affect scalability

Problem:You have a machine learning model that works well on a small dataset but becomes very slow and uses too much memory when the dataset grows larger.

Current Metrics:Training time per epoch: 5 minutes on 10,000 samples; Memory usage: 4 GB; Validation accuracy: 88%

Issue:The model architecture is too complex and not optimized for larger datasets, causing slow training and high memory use, limiting scalability.

Your Task

Modify the model architecture to reduce training time and memory usage while keeping validation accuracy above 85%.

Do not reduce the dataset size.

Keep the same training and validation split.

Do not change the optimizer or learning rate.

Hint 1

Hint 2

Hint 3

Solution

Prompt Engineering / GenAI

import tensorflow as tf
from tensorflow.keras import layers, models

# Original complex model
# model = models.Sequential([
#     layers.Dense(512, activation='relu', input_shape=(input_dim,)),
#     layers.Dense(512, activation='relu'),
#     layers.Dense(256, activation='relu'),
#     layers.Dense(num_classes, activation='softmax')
# ])

# Simplified model for better scalability
model = models.Sequential([
    layers.Dense(128, activation='relu', input_shape=(input_dim,)),
    layers.Dropout(0.3),
    layers.Dense(64, activation='relu'),
    layers.BatchNormalization(),
    layers.Dense(num_classes, activation='softmax')
])

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# Assuming X_train, y_train, X_val, y_val are predefined
history = model.fit(X_train, y_train, epochs=20, batch_size=64, validation_data=(X_val, y_val))

Reduced the number of units per layer from 512/512/256 to 128/64.

Added dropout layer with rate 0.3 to reduce overfitting.

Added batch normalization to stabilize training.

Kept the same optimizer and learning rate.

Results Interpretation

Before: Training time 5 min/epoch, Memory 4 GB, Validation accuracy 88%

After: Training time 2 min/epoch, Memory 2 GB, Validation accuracy 86%

Simplifying model architecture by reducing layers and units, and adding dropout and batch normalization, can greatly improve scalability by lowering training time and memory use while maintaining good accuracy.

Bonus Experiment

Try using a convolutional neural network (CNN) architecture instead of a fully connected network to improve scalability on image data.

💡 Hint

CNNs use shared weights and local connections, which reduce parameters and improve efficiency on images.