Why architecture choices affect scalability in Prompt Engineering / GenAI - Challenge Your Understanding

Challenge - 5 Problems

🧠 Conceptual

intermediate

How does model architecture impact training speed?

Imagine you have two models: one with many layers and one with fewer layers. Which statement best explains how the number of layers affects training speed?

A More layers usually mean slower training because there are more calculations to do.

B More layers always make training faster because the model learns better.

C The number of layers does not affect training speed at all.

D Fewer layers always cause the model to crash during training.

💡 Hint

Think about how adding more steps in a recipe takes more time.

More layers add more calculations, so training takes longer. This affects how scalable the model is when using bigger data or limited hardware.

❓ Model Choice

intermediate

Choosing architecture for large datasets

You have a very large dataset and limited computing power. Which model architecture choice helps scalability best?

A A model that requires loading the entire dataset into memory.

B A very deep neural network with millions of parameters.

C A simple model with fewer layers and parameters.

D A model that uses all available memory at once.

💡 Hint

Think about what works well when your computer has limited memory.

Simpler models with fewer parameters use less memory and compute, making them more scalable on limited hardware.

❓ Hyperparameter

advanced

Effect of batch size on scalability

Consider training a model with different batch sizes. Which batch size choice best supports scalability on limited GPU memory?

A Batch size equal to the entire dataset size.

B Very small batch size that fits comfortably in GPU memory.

C Batch size does not affect GPU memory usage.

D Very large batch size that uses all GPU memory at once.

💡 Hint

Think about how much data you can hold in your hands at once.

Smaller batch sizes use less memory, allowing training on limited GPUs and improving scalability.

❓ Metrics

advanced

Measuring scalability with training time

You train two models on the same dataset. Model A takes 2 hours, Model B takes 6 hours. Both have similar accuracy. What does this say about their scalability?

A Model B is more scalable because it takes longer to train.

B Training time does not relate to scalability.

C Both models have the same scalability because accuracy is similar.

D Model A is more scalable because it trains faster with similar accuracy.

💡 Hint

Think about which model can handle bigger data faster.

Faster training with similar accuracy means better scalability, as the model can handle larger data or more experiments efficiently.

🔧 Debug

expert

Identifying architecture bottleneck in scalability

Given a model that slows down drastically when data size doubles, which architectural choice is most likely causing the bottleneck?

A Using a fully connected layer with very large input size causing many computations.

B Using convolutional layers with small filters and stride 1.

C Using batch normalization layers after each activation.

D Using dropout layers to reduce overfitting.

💡 Hint

Think about which layer type grows computation most with input size.

Fully connected layers scale poorly with input size because they connect every input to every neuron, causing a big increase in computations as data grows.