Cost optimization in Prompt Engineering / GenAI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Cost Components in Machine Learning

Which of the following is NOT typically considered a direct cost component when optimizing machine learning model deployment?

A Developer salaries for model development

B Compute resources used during model training

C Electricity costs for powering data center cooling

D Data storage costs for training datasets

💡 Hint

Think about costs directly tied to running and maintaining the model versus indirect overhead.

Developer salaries are indirect operational costs, not direct costs of model deployment or training.

❓ Model Choice

intermediate

Choosing a Model for Cost Efficiency

You need to deploy a model on edge devices with limited compute and power. Which model type is the most cost-efficient choice?

A Small convolutional neural network with quantization applied

B Uncompressed recurrent neural network with high precision weights

C Ensemble of multiple deep neural networks

D Large transformer-based model with billions of parameters

💡 Hint

Consider model size and computational requirements for edge deployment.

Small CNNs with quantization reduce model size and computation, lowering cost on edge devices.

❓ Metrics

advanced

Evaluating Cost-Performance Tradeoff

Given two models with the following metrics:

• Model A: Accuracy 92%, Inference cost $0.10 per 1000 predictions
• Model B: Accuracy 90%, Inference cost $0.02 per 1000 predictions

Which metric best helps decide the cost-effectiveness of these models?

A Inference cost alone

B Accuracy divided by inference cost

C Accuracy alone

D Training time

💡 Hint

Think about combining accuracy and cost into one measure.

Dividing accuracy by cost gives a ratio showing performance per cost unit, aiding cost-effectiveness decisions.

🔧 Debug

advanced

Identifying Cost Bottleneck in Model Training Code

Consider this Python snippet for training a model:

for epoch in range(10):
    for batch in data_loader:
        outputs = model(batch)
        loss = loss_fn(outputs, batch.labels)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        time.sleep(5)

What is the main cause of unnecessary cost increase?

A Not using GPU acceleration

B Missing learning rate scheduler

C The time.sleep(5) call inside the training loop

D Not shuffling data_loader

💡 Hint

Look for code that delays training without benefit.

The time.sleep(5) pauses training each batch, wasting compute time and increasing cost.

❓ Hyperparameter

expert

Optimizing Hyperparameters for Cost Reduction

You want to reduce training cost by adjusting batch size and learning rate. Which combination is most likely to reduce cost without hurting model convergence?

A Decrease batch size and decrease learning rate

B Decrease batch size and increase learning rate

C Increase batch size and decrease learning rate

D Increase batch size and increase learning rate

💡 Hint

Think about how batch size affects training speed and learning rate affects convergence.

Increasing batch size reduces number of updates, and increasing learning rate can maintain convergence speed, lowering cost.