Evaluation metrics (accuracy, F1, confusion matrix) in NLP - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to calculate accuracy score for predictions.

NLP

from sklearn.metrics import [1]

true_labels = [0, 1, 1, 0, 1]
pred_labels = [0, 0, 1, 0, 1]

acc = [1](true_labels, pred_labels)
print(f"Accuracy: {acc}")

Drag options to blanks, or click blank then click option'

Aconfusion_matrix

Bf1_score

Caccuracy_score

Dclassification_report

Attempts:

3 left

💡 Hint

Common Mistakes

Using f1_score instead of accuracy_score for accuracy calculation.

Confusing confusion_matrix output with accuracy value.

The accuracy_score function calculates the ratio of correct predictions to total predictions.

2fill in blank

medium

Complete the code to calculate F1 score for binary classification.

NLP

from sklearn.metrics import f1_score

true_labels = [1, 0, 1, 1, 0]
pred_labels = [1, 0, 0, 1, 0]

f1 = f1_score(true_labels, pred_labels, [1]='binary')
print(f"F1 Score: {f1}")

Drag options to blanks, or click blank then click option'

Aaverage

Bpos_label

Clabels

Dzero_division

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'pos_label' instead of 'average' parameter.

Omitting the 'average' parameter causing errors in multiclass.

The 'average' parameter specifies the type of averaging performed on the data for F1 score calculation.

3fill in blank

hard

Fix the error in the code to correctly compute confusion matrix.

NLP

from sklearn.metrics import confusion_matrix

true = [1, 0, 1, 1, 0]
pred = [1, 1, 1, 0, 0]

cm = confusion_matrix([1], pred)
print(cm)

Drag options to blanks, or click blank then click option'

Atrue

Bpred

Caverage

Dlabels

Attempts:

3 left

💡 Hint

Common Mistakes

Swapping true and predicted labels in confusion_matrix function.

Passing parameters unrelated to labels.

The first argument to confusion_matrix must be the true labels, followed by predicted labels.

4fill in blank

hard

Fill both blanks to create a confusion matrix and extract true positives.

NLP

from sklearn.metrics import confusion_matrix

true = [0, 1, 0, 1, 1]
pred = [0, 0, 0, 1, 1]

cm = confusion_matrix(true, pred)
true_positives = cm[1][2]
print(f"True Positives: {true_positives}")

Drag options to blanks, or click blank then click option'

A[1]

B[1, 1]

C[0, 0]

D[0]

Attempts:

3 left

💡 Hint

Common Mistakes

Using cm[0][0] which is true negatives, not true positives.

Using incorrect indices causing index errors.

True positives are located at row 1, column 1 in the confusion matrix, accessed by cm[1][1].

5fill in blank

hard

Fill all three blanks to compute accuracy, F1 score, and confusion matrix for predictions.

NLP

from sklearn.metrics import [1], [2], [3]

true = [1, 0, 1, 0, 1]
pred = [1, 0, 0, 0, 1]

acc = [1](true, pred)
f1 = [2](true, pred, average='binary')
cm = [3](true, pred)

print(f"Accuracy: {acc}")
print(f"F1 Score: {f1}")
print(f"Confusion Matrix:\n{cm}")

Drag options to blanks, or click blank then click option'

Aaccuracy_score

Bf1_score

Cconfusion_matrix

Dclassification_report

Attempts:

3 left

💡 Hint

Common Mistakes

Mixing up metric function names or parameters.

Forgetting to specify average='binary' for f1_score.

Use accuracy_score for accuracy, f1_score for F1, and confusion_matrix for confusion matrix.

Practice

1. What does the accuracy metric measure in a classification model?

easy

A. The proportion of correct predictions out of all predictions

B. The balance between precision and recall

C. The number of false positives only

D. The total number of classes in the dataset

Solution

1. Step 1: Understand accuracy definition

   Accuracy is defined as the number of correct predictions divided by the total number of predictions made.

2. Step 2: Compare options with definition

   Only The proportion of correct predictions out of all predictions correctly describes accuracy as the proportion of correct predictions out of all predictions.

3. Final Answer:

   The proportion of correct predictions out of all predictions -> Option A

4. Quick Check:

   Accuracy = Correct predictions / Total predictions [OK]

Hint: Accuracy = correct predictions divided by total predictions [OK]

Common Mistakes:

• Confusing accuracy with F1 score
• Thinking accuracy measures only false positives
• Believing accuracy counts number of classes

2. Which of the following is the correct formula for F1 score?

easy

A. Precision + Recall

B. 2 * (Precision * Recall) / (Precision + Recall)

C. True Positives / Total Samples

D. True Negatives / (True Negatives + False Positives)

Solution

1. Step 1: Recall F1 score formula

   F1 score is the harmonic mean of precision and recall, calculated as 2 times their product divided by their sum.

2. Step 2: Match formula with options

   2 * (Precision * Recall) / (Precision + Recall) matches the correct formula: 2 * (Precision * Recall) / (Precision + Recall).

3. Final Answer:

   2 * (Precision * Recall) / (Precision + Recall) -> Option B

4. Quick Check:

   F1 = 2PR/(P+R) [OK]

Hint: F1 score = 2 * Precision * Recall / (Precision + Recall) [OK]

Common Mistakes:

• Adding precision and recall instead of harmonic mean
• Using true positives over total samples as F1
• Confusing F1 with specificity

3. Given the confusion matrix below for a binary classifier:

[[50, 10],
 [5, 35]]

What is the accuracy of the model?

medium

A. 75%

B. 70%

C. 90%

D. 85%

Solution

1. Step 1: Identify confusion matrix values

   True Positives (TP) = 50, False Positives (FP) = 10, False Negatives (FN) = 5, True Negatives (TN) = 35.

2. Step 2: Calculate accuracy

   Accuracy = (TP + TN) / (TP + FP + FN + TN) = (50 + 35) / (50 + 10 + 5 + 35) = 85 / 100 = 0.85 or 85%.

3. Final Answer:

   85% -> Option D

4. Quick Check:

   Accuracy = (TP+TN)/Total = 85/100 = 85% [OK]

Hint: Accuracy = (TP + TN) / total samples [OK]

Common Mistakes:

• Adding false positives or false negatives to numerator
• Calculating only TP / total samples
• Mixing up TP and TN values

4. You have this confusion matrix:

[[40, 20],
 [10, 30]]

Which line of code correctly calculates precision for the positive class?

medium

A. precision = TP / (TP + FP)

B. precision = TP / (TP + FN)

C. precision = TN / (TN + FP)

D. precision = TP / (TP + TN)

Solution

1. Step 1: Recall precision formula

   Precision is the ratio of true positives to all predicted positives: TP / (TP + FP).

2. Step 2: Match formula with options

   precision = TP / (TP + FP) correctly uses TP / (TP + FP). precision = TP / (TP + FN) uses recall formula, C and D are incorrect.

3. Final Answer:

   precision = TP / (TP + FP) -> Option A

4. Quick Check:

   Precision = TP / (TP + FP) [OK]

Hint: Precision = true positives / predicted positives [OK]

Common Mistakes:

• Using TP / (TP + FN) which is recall
• Confusing TN with TP in precision
• Dividing by TP + TN instead of TP + FP

5. A model has precision = 0.8 and recall = 0.5. What is the F1 score? Choose the closest value.

hard

A. 0.70

B. 0.65

C. 0.62

D. 0.75

Solution

1. Step 1: Recall F1 score formula

   F1 = 2 * (Precision * Recall) / (Precision + Recall) = 2 * (0.8 * 0.5) / (0.8 + 0.5).

2. Step 2: Calculate F1 score

   Calculate numerator: 2 * 0.4 = 0.8. Calculate denominator: 1.3. F1 = 0.8 / 1.3 ≈ 0.615.

3. Final Answer:

   0.62 -> Option C

4. Quick Check:

   F1 ≈ 0.62 from 0.8 precision and 0.5 recall [OK]

Hint: F1 is harmonic mean: 2PR/(P+R), plug values carefully [OK]

Common Mistakes:

• Averaging precision and recall instead of harmonic mean
• Mixing up precision and recall values
• Rounding too early causing wrong final answer