Error rate and failure analysis in Agentic AI - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to calculate the error rate given the number of errors and total samples.

Agentic AI

error_rate = errors / [1]

Drag options to blanks, or click blank then click option'

Atotal_samples

Bcorrect_predictions

Cerrors

Dpredictions

Attempts:

3 left

💡 Hint

Common Mistakes

Dividing errors by errors instead of total samples

Using correct predictions as denominator

The error rate is calculated by dividing the number of errors by the total number of samples.

2fill in blank

medium

Complete the code to compute the failure rate as a percentage.

Agentic AI

failure_rate = (failures / total_tests) * [1]

Drag options to blanks, or click blank then click option'

A10

B0.01

C100

D1

Attempts:

3 left

💡 Hint

Common Mistakes

Multiplying by 0.01 instead of 100

Forgetting to multiply at all

To convert a fraction to a percentage, multiply by 100.

3fill in blank

hard

Fix the error in the code to calculate the failure count from error rate and total attempts.

Agentic AI

failure_count = [1] * total_attempts

Drag options to blanks, or click blank then click option'

Asuccess_rate

Berror_rate

Ctotal_attempts

Dfailure_rate

Attempts:

3 left

💡 Hint

Common Mistakes

Using total_attempts instead of error_rate

Using success_rate which is the opposite

The failure count is the error rate multiplied by the total number of attempts.

4fill in blank

hard

Fill both blanks to create a dictionary that maps test names to their failure counts only if failures are greater than zero.

Agentic AI

failure_dict = {test: [1] for test, failures in test_results.items() if failures [2] 0}

Drag options to blanks, or click blank then click option'

Afailures

B>

C==

D<

Attempts:

3 left

💡 Hint

Common Mistakes

Using equality instead of greater than

Using test name instead of failure count as value

The dictionary comprehension includes only tests with failures greater than zero, mapping test names to failure counts.

5fill in blank

hard

Fill all three blanks to create a dictionary of failure rates for tests with failures above threshold.

Agentic AI

failure_rates = {test: failures / [1] for test, failures in test_results.items() if failures [2] threshold and failures [3] 0}

Drag options to blanks, or click blank then click option'

Atotal_tests

B>

C!=

Dtotal_attempts

Attempts:

3 left

💡 Hint

Common Mistakes

Using total_tests instead of total_attempts

Using equality instead of inequality in conditions

The failure rate is failures divided by total attempts. The dictionary includes tests where failures are greater than the threshold and not zero.

Practice

1. What does the error rate in a machine learning model represent?

easy

A. The percentage of wrong predictions made by the model

B. The time taken to train the model

C. The number of features used in the model

D. The size of the training dataset

Solution

1. Step 1: Understand what error rate measures

   Error rate measures how often the model's predictions are incorrect compared to the true answers.

2. Step 2: Relate error rate to model performance

   A higher error rate means more wrong predictions, so it shows the model's mistakes.

3. Final Answer:

   The percentage of wrong predictions made by the model -> Option A

4. Quick Check:

   Error rate = wrong predictions percentage [OK]

Hint: Error rate means how often the model is wrong [OK]

Common Mistakes:

• Confusing error rate with training time
• Thinking error rate counts features
• Mixing error rate with dataset size

2. Which of the following is the correct way to calculate error rate given total_predictions and wrong_predictions?

easy

A. error_rate = total_predictions / wrong_predictions

B. error_rate = total_predictions - wrong_predictions

C. error_rate = wrong_predictions * total_predictions

D. error_rate = wrong_predictions / total_predictions

Solution

1. Step 1: Recall error rate formula

   Error rate is the fraction of wrong predictions out of all predictions made.

2. Step 2: Match formula to options

   error_rate = wrong_predictions / total_predictions correctly divides wrong predictions by total predictions to get error rate.

3. Final Answer:

   error_rate = wrong_predictions / total_predictions -> Option D

4. Quick Check:

   Error rate = wrong / total [OK]

Hint: Divide wrong predictions by total predictions [OK]

Common Mistakes:

• Reversing numerator and denominator
• Multiplying instead of dividing
• Subtracting counts instead of dividing

3. Given the following code, what is the printed error rate?

total = 100
wrong = 7
error_rate = wrong / total
print(f"Error rate: {error_rate:.2f}")

medium

A. Error rate: 7.00

B. Error rate: 0.07

C. Error rate: 0.70

D. Error rate: 0.007

Solution

1. Step 1: Calculate error rate value

   error_rate = 7 / 100 = 0.07

2. Step 2: Format output to 2 decimals

   Formatted as 0.07 in the print statement.

3. Final Answer:

   Error rate: 0.07 -> Option B

4. Quick Check:

   7 divided by 100 = 0.07 [OK]

Hint: Divide wrong by total and format to two decimals [OK]

Common Mistakes:

• Confusing 7% with 7.0
• Multiplying instead of dividing
• Misreading decimal places

4. A model's error rate is unexpectedly high. Which of the following is the best first step in failure analysis?

medium

A. Check the data for incorrect labels or noise

B. Increase the number of training epochs immediately

C. Add more layers to the model without checking data

D. Reduce the size of the test dataset

Solution

1. Step 1: Understand failure analysis purpose

   Failure analysis looks for root causes of errors, often starting with data quality.

2. Step 2: Evaluate options for best first step

   Checking data labels or noise is the most direct way to find why errors happen.

3. Final Answer:

   Check the data for incorrect labels or noise -> Option A

4. Quick Check:

   Start failure analysis by checking data quality [OK]

Hint: Start failure analysis by checking data quality [OK]

Common Mistakes:

• Jumping to model changes without data check
• Ignoring data errors as cause
• Changing test set size instead of fixing errors

5. You have a model with 10,000 predictions and 500 errors. After failure analysis, you find 200 errors caused by mislabeled data. What is the corrected error rate after fixing labels?

hard

A. 0.07

B. 0.05

C. 0.03

D. 0.02

Solution

1. Step 1: Calculate original error rate

   Original errors = 500, total = 10,000, so error rate = 500/10,000 = 0.05

2. Step 2: Remove errors due to mislabeled data

   Corrected errors = 500 - 200 = 300

3. Step 3: Calculate corrected error rate

   Corrected error rate = 300 / 10,000 = 0.03

4. Final Answer:

   0.03 -> Option C

5. Quick Check:

   (500-200)/10000 = 0.03 [OK]

Hint: Subtract mislabeled errors before dividing [OK]

Common Mistakes:

• Not removing mislabeled errors
• Dividing mislabeled errors by total
• Adding mislabeled errors instead of subtracting