Cost optimization in Prompt Engineering / GenAI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Cost Components in Machine Learning

Which of the following is NOT typically considered a direct cost component when optimizing machine learning model deployment?

A Developer salaries for model development

B Compute resources used during model training

C Electricity costs for powering data center cooling

D Data storage costs for training datasets

💡 Hint

Think about costs directly tied to running and maintaining the model versus indirect overhead.

Developer salaries are indirect operational costs, not direct costs of model deployment or training.

❓ Model Choice

intermediate

Choosing a Model for Cost Efficiency

You need to deploy a model on edge devices with limited compute and power. Which model type is the most cost-efficient choice?

A Small convolutional neural network with quantization applied

B Uncompressed recurrent neural network with high precision weights

C Ensemble of multiple deep neural networks

D Large transformer-based model with billions of parameters

💡 Hint

Consider model size and computational requirements for edge deployment.

Small CNNs with quantization reduce model size and computation, lowering cost on edge devices.

❓ Metrics

advanced

Evaluating Cost-Performance Tradeoff

Given two models with the following metrics:

• Model A: Accuracy 92%, Inference cost $0.10 per 1000 predictions
• Model B: Accuracy 90%, Inference cost $0.02 per 1000 predictions

Which metric best helps decide the cost-effectiveness of these models?

A Inference cost alone

B Accuracy divided by inference cost

C Accuracy alone

D Training time

💡 Hint

Think about combining accuracy and cost into one measure.

Dividing accuracy by cost gives a ratio showing performance per cost unit, aiding cost-effectiveness decisions.

🔧 Debug

advanced

Identifying Cost Bottleneck in Model Training Code

Consider this Python snippet for training a model:

for epoch in range(10):
    for batch in data_loader:
        outputs = model(batch)
        loss = loss_fn(outputs, batch.labels)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        time.sleep(5)

What is the main cause of unnecessary cost increase?

A Not using GPU acceleration

B Missing learning rate scheduler

C The time.sleep(5) call inside the training loop

D Not shuffling data_loader

💡 Hint

Look for code that delays training without benefit.

The time.sleep(5) pauses training each batch, wasting compute time and increasing cost.

❓ Hyperparameter

expert

Optimizing Hyperparameters for Cost Reduction

You want to reduce training cost by adjusting batch size and learning rate. Which combination is most likely to reduce cost without hurting model convergence?

A Decrease batch size and decrease learning rate

B Decrease batch size and increase learning rate

C Increase batch size and decrease learning rate

D Increase batch size and increase learning rate

💡 Hint

Think about how batch size affects training speed and learning rate affects convergence.

Increasing batch size reduces number of updates, and increasing learning rate can maintain convergence speed, lowering cost.

Practice

1.

What is the main goal of cost optimization in machine learning?

easy

A. To reduce expenses while keeping good model accuracy

B. To make the model as large as possible

C. To use all available data regardless of cost

D. To increase training time for better results

Solution

1. Step 1: Understand cost optimization meaning

   Cost optimization means saving money and resources in AI work.

2. Step 2: Connect cost saving with accuracy

   Good cost optimization keeps accuracy high while lowering expenses.

3. Final Answer:

   To reduce expenses while keeping good model accuracy -> Option A

4. Quick Check:

   Cost optimization = reduce cost + keep accuracy [OK]

Hint: Cost optimization balances cost and accuracy [OK]

Common Mistakes:

• Thinking bigger models always mean better cost
• Ignoring accuracy when saving cost
• Assuming more data always reduces cost

2.

Which of the following is the correct way to reduce training cost in AI?

options = [
  'Use smaller models',
  'Train on all data without filtering',
  'Increase batch size unnecessarily',
  'Use slower hardware'
]

easy

A. Use slower hardware

B. Train on all data without filtering

C. Use smaller models

D. Increase batch size unnecessarily

Solution

1. Step 1: Identify cost-saving methods

   Using smaller models reduces computation and memory, lowering cost.

2. Step 2: Evaluate other options

   Training on all data, increasing batch size unnecessarily, or using slower hardware increase cost or slow training.

3. Final Answer:

   Use smaller models -> Option C

4. Quick Check:

   Smaller models reduce cost [OK]

Hint: Smaller models usually cost less to train [OK]

Common Mistakes:

• Thinking more data always reduces cost
• Believing bigger batch size always helps
• Assuming slower hardware saves money

3.

Consider this Python code that trains a model with different batch sizes to optimize cost:

batch_sizes = [16, 32, 64]
costs = []
for b in batch_sizes:
    cost = 1000 / b  # cost inversely proportional to batch size
    costs.append(cost)
print(costs)

What is the output of this code?

medium

A. [64, 32, 16]

B. [16, 32, 64]

C. [15.625, 31.25, 62.5]

D. [62.5, 31.25, 15.625]

Solution

1. Step 1: Calculate cost for each batch size

   For batch size 16: 1000/16 = 62.5; for 32: 1000/32 = 31.25; for 64: 1000/64 = 15.625.

2. Step 2: Collect costs in list and print

   The costs list becomes [62.5, 31.25, 15.625], which is printed.

3. Final Answer:

   [62.5, 31.25, 15.625] -> Option D

4. Quick Check:

   Cost = 1000 / batch size [OK]

Hint: Divide 1000 by each batch size to get costs [OK]

Common Mistakes:

• Confusing batch sizes with costs
• Mixing up division order
• Copying batch_sizes list instead of costs

4.

Find the error in this code snippet that tries to reduce training cost by skipping data points:

data = [1, 2, 3, 4, 5]
reduced_data = [x for x in data if x > 3]
print(reduced_data)

What is the problem if the goal is to keep most data but reduce cost?

medium

A. It removes too many data points, hurting accuracy

B. It does not remove any data points

C. It causes a syntax error

D. It duplicates data points

Solution

1. Step 1: Understand filtering condition

   The code keeps only data points greater than 3, removing 1, 2, 3.

2. Step 2: Assess impact on data and cost

   Removing many points reduces data but may hurt model accuracy since much data is lost.

3. Final Answer:

   It removes too many data points, hurting accuracy -> Option A

4. Quick Check:

   Filtering >3 removes many points [OK]

Hint: Check how much data filtering removes [OK]

Common Mistakes:

• Thinking it keeps most data
• Expecting syntax error
• Assuming data duplicates

5.

You want to optimize cost for training a language model. You have these options:

• Use a smaller model
• Train on a filtered smaller dataset
• Use mixed precision training
• Train longer with bigger batch size

Which combination best balances cost and accuracy?

hard

A. Train longer with bigger batch size only

B. Use smaller model + filtered dataset + mixed precision

C. Use smaller model only

D. Train on full dataset with no precision changes

Solution

1. Step 1: Analyze each option's effect on cost and accuracy

   Smaller model reduces cost; filtered dataset reduces data size; mixed precision speeds training and saves memory.

2. Step 2: Combine options for best balance

   Using all three together lowers cost while keeping good accuracy. Training longer with bigger batch size alone increases cost.

3. Final Answer:

   Use smaller model + filtered dataset + mixed precision -> Option B

4. Quick Check:

   Combine cost-saving methods for best results [OK]

Hint: Combine multiple cost-saving methods for best effect [OK]

Common Mistakes:

• Choosing only one method
• Ignoring accuracy impact
• Assuming longer training always helps