Environmental impact of AI in Prompt Engineering / GenAI - Model Metrics & Evaluation

Metrics & Evaluation - Environmental impact of AI

Which metric matters for this concept and WHY

For understanding the environmental impact of AI, the key metrics are energy consumption and carbon footprint. These show how much electricity AI models use and how much greenhouse gas is released. Tracking these helps us know if AI is using resources wisely and if it harms the planet.

Confusion matrix or equivalent visualization (ASCII)

Energy Use (kWh) | Carbon Emissions (kg CO2)
------------------------------------------
Training Large Model   |  1000 kWh  |  500 kg
Training Small Model   |   100 kWh  |   50 kg
Inference per Request  |  0.01 kWh  | 0.005 kg
------------------------------------------
Total AI Usage         |  1100.01 kWh  |  550.005 kg
    

This table shows energy and emissions for different AI tasks. It helps compare how big or small models impact the environment.

Precision vs Recall (or equivalent tradeoff) with concrete examples

Here, the tradeoff is between model performance and environmental cost. For example, a very large AI model may give better answers (higher accuracy) but use much more energy. A smaller model uses less energy but may be less accurate.

Choosing the right balance means picking a model that is good enough but does not waste energy. This is like choosing a car that is fast but also fuel efficient.

What "good" vs "bad" metric values look like for this use case

Good: AI models that use less than 100 kWh for training and keep carbon emissions low, while still performing well. Efficient code and hardware help.

Bad: Models that use thousands of kWh and emit hundreds of kg CO2 for small improvements in accuracy. This wastes energy and harms the environment.

Metrics pitfalls (accuracy paradox, data leakage, overfitting indicators)

One pitfall is ignoring environmental metrics and focusing only on accuracy. A model can be very accurate but cause huge energy waste.

Another is not measuring energy use consistently, leading to wrong conclusions about impact.

Also, overfitting large models wastes energy training on data that does not improve real-world use.

Your model has 98% accuracy but 12% recall on fraud. Is it good?

This question is about fraud detection, not environmental impact, but it shows tradeoffs. High accuracy with low recall means many fraud cases are missed.

For environmental impact, a similar question is: "Is a model that uses 1000 kWh worth a 1% accuracy gain over a model using 100 kWh?" Usually, no, because the environmental cost is too high for small benefit.

Key Result

Energy consumption and carbon footprint are key metrics to evaluate AI's environmental impact and balance performance with sustainability.

Practice

1. What is the main environmental concern related to training large AI models?

easy

A. AI models increasing water pollution

B. AI models causing deforestation directly

C. AI models producing plastic waste

D. High energy consumption leading to increased carbon emissions

Solution

1. Step 1: Understand AI training process

   Training large AI models requires a lot of computer power, which uses electricity.

2. Step 2: Link electricity use to environmental impact

   Electricity often comes from burning fossil fuels, which releases carbon emissions harming the environment.

3. Final Answer:

   High energy consumption leading to increased carbon emissions -> Option D

4. Quick Check:

   Energy use = Carbon emissions [OK]

Hint: Think about what powers computers during training [OK]

Common Mistakes:

• Confusing AI's indirect impact with direct pollution
• Thinking AI models produce physical waste
• Ignoring energy source in environmental impact

2. Which of the following is the correct way to reduce AI's environmental impact?

easy

A. Use larger models with more layers

B. Train models using renewable energy sources

C. Increase training time without optimization

D. Ignore energy consumption during model design

Solution

1. Step 1: Identify methods to reduce carbon footprint

   Using renewable energy like solar or wind reduces carbon emissions from electricity.

2. Step 2: Evaluate options for environmental friendliness

   Options A, B, and D increase energy use or ignore it, so they don't reduce impact.

3. Final Answer:

   Train models using renewable energy sources -> Option B

4. Quick Check:

   Renewable energy = Lower carbon footprint [OK]

Hint: Choose options that lower energy or use clean energy [OK]

Common Mistakes:

• Thinking bigger models always help
• Ignoring energy source in training
• Assuming longer training is better for environment

3. Consider this code snippet estimating AI model energy use:

energy_per_epoch = 50  # kWh
epochs = 10
carbon_per_kwh = 0.4  # kg CO2
carbon_footprint = energy_per_epoch * epochs * carbon_per_kwh
print(carbon_footprint)

What is the output of this code?

medium

A. 200.0

B. 500.0

C. 20.0

D. 400.0

Solution

1. Step 1: Calculate total energy used

   Energy per epoch (50 kWh) times epochs (10) equals 500 kWh total.

2. Step 2: Calculate carbon footprint

   Multiply total energy (500 kWh) by carbon per kWh (0.4 kg CO2) = 200 kg CO2.

3. Final Answer:

   200.0 -> Option A

4. Quick Check:

   50 * 10 * 0.4 = 200.0 [OK]

Hint: Multiply energy, epochs, and carbon per kWh [OK]

Common Mistakes:

• Multiplying incorrectly or missing one factor
• Confusing units or decimal points
• Mixing up variable names

4. This code tries to calculate carbon footprint but has a bug:

energy_per_epoch = 40
epochs = '10'
carbon_per_kwh = 0.3
carbon_footprint = energy_per_epoch * epochs * carbon_per_kwh
print(carbon_footprint)

What is the error and how to fix it?

medium

A. SyntaxError due to missing colon

B. NameError because carbon_per_kwh is undefined

C. TypeError because epochs is a string; convert it to int

D. No error; code runs fine

Solution

1. Step 1: Identify variable types

   epochs is a string '10', but multiplication needs a number.

2. Step 2: Fix type mismatch

   Convert epochs to integer using int(epochs) to allow multiplication.

3. Final Answer:

   TypeError because epochs is a string; convert it to int -> Option C

4. Quick Check:

   String * float causes error [OK]

Hint: Check variable types before math operations [OK]

Common Mistakes:

• Ignoring type mismatch errors
• Assuming code runs without conversion
• Confusing error types

5. You want to reduce the environmental impact of an AI project. Which combined approach is best?

hard

A. Use smaller models, train fewer epochs, and power training with renewable energy

B. Use larger models, train longer, and use coal-based electricity

C. Ignore model size, focus only on data quality

D. Train models on any energy source but optimize only accuracy

Solution

1. Step 1: Identify factors affecting environmental impact

   Model size, training time, and energy source all affect energy use and emissions.

2. Step 2: Combine best practices

   Smaller models and fewer epochs reduce energy use; renewable energy lowers carbon footprint.

3. Final Answer:

   Use smaller models, train fewer epochs, and power training with renewable energy -> Option A

4. Quick Check:

   Smaller + less training + clean energy = less impact [OK]

Hint: Combine smaller models, less training, and clean energy [OK]

Common Mistakes:

• Focusing on accuracy only
• Ignoring energy source
• Assuming bigger models are better for environment