Error rate and failure analysis in Agentic AI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Error Rate in Classification

Which of the following best describes the error rate in a classification model?

A The proportion of all predictions that are incorrect out of the total predictions made.

B The proportion of correct predictions out of the total predictions made.

C The number of true positives divided by false negatives.

D The ratio of false positives to true negatives.

💡 Hint

Think about what 'error rate' means in everyday terms: how often does the model get it wrong?

Error rate measures how often the model makes wrong predictions compared to all predictions. It is the complement of accuracy.

❓ Metrics

intermediate

Calculating Failure Rate from Confusion Matrix

Given a confusion matrix for a binary classifier:
True Positives = 80, True Negatives = 90, False Positives = 10, False Negatives = 20.
What is the failure rate (error rate) of this model?

A 0.10

B 0.20

C 0.15

D 0.25

💡 Hint

Failure rate = (False Positives + False Negatives) / Total predictions.

Total predictions = 80 + 90 + 10 + 20 = 200.
Failures = 10 + 20 = 30.
Failure rate = 30 / 200 = 0.15.

🔧 Debug

advanced

Identifying the Cause of High Error Rate

A model trained on a dataset shows a high error rate on the test set but low error rate on the training set. What is the most likely cause?

A The model is underfitting and needs more training epochs.

B The model is overfitting the training data and failing to generalize.

C The test set is too small to measure error rate accurately.

D The training data contains too many missing values.

💡 Hint

Think about what it means when training error is low but test error is high.

Overfitting means the model learned the training data too well, including noise, so it performs poorly on new data.

❓ Model Choice

advanced

Choosing a Model to Reduce Failure Rate on Imbalanced Data

You have a dataset with 95% of one class and 5% of another. Which model or technique is best to reduce failure rate on the minority class?

A Use a decision tree with balanced class weights or oversampling of minority class.

B Use a clustering algorithm like K-means.

C Use a linear regression model.

D Use a simple logistic regression without class weighting.

💡 Hint

Think about how to handle imbalanced classes to reduce errors on the minority group.

Balanced class weights or oversampling help the model pay more attention to the minority class, reducing its error rate.

❓ Predict Output

expert

Output of Failure Rate Calculation Code

What is the output of this Python code that calculates failure rate?

Agentic AI

def failure_rate(conf_matrix):
    TP, TN, FP, FN = conf_matrix
    total = TP + TN + FP + FN
    return (FP + FN) / total

conf_matrix = (50, 40, 5, 5)
print(failure_rate(conf_matrix))

A 0.05

B 0.15

C 0.20

D 0.10

💡 Hint

Calculate (FP + FN) / total using the given numbers.

FP + FN = 5 + 5 = 10, total = 50 + 40 + 5 + 5 = 100, so failure rate = 10 / 100 = 0.10.