Why LLM evaluation ensures quality in Prompt Engineering / GenAI - Test Your Understanding

Practice - 5 Tasks

1fill in blank

easy

Complete the code to print the accuracy of the LLM predictions.

Prompt Engineering / GenAI

accuracy = sum(predictions == labels) / len(labels)
print('Accuracy:', [1])

Drag options to blanks, or click blank then click option'

Aaccuracy

Bpredictions

Clabels

Dlen(predictions)

Attempts:

3 left

💡 Hint

Common Mistakes

Printing predictions or labels instead of accuracy.

Using length of predictions instead of accuracy.

The accuracy variable holds the ratio of correct predictions to total labels, so printing accuracy shows the model's quality.

2fill in blank

medium

Complete the code to calculate the F1 score for LLM evaluation.

Prompt Engineering / GenAI

from sklearn.metrics import f1_score
f1 = f1_score(labels, [1])
print('F1 Score:', f1)

Drag options to blanks, or click blank then click option'

Alabels

Bscores

Caccuracy

Dpredictions

Attempts:

3 left

💡 Hint

Common Mistakes

Passing labels twice instead of predictions.

Passing accuracy or scores which are not label arrays.

The f1_score function compares true labels with predicted labels to compute the F1 score, so predictions is the correct input.

3fill in blank

hard

Fix the error in the code to compute the confusion matrix for LLM outputs.

Prompt Engineering / GenAI

from sklearn.metrics import confusion_matrix
cm = confusion_matrix(labels, [1])
print(cm)

Drag options to blanks, or click blank then click option'

Alabels

Bpredictions

Caccuracy

Dscores

Attempts:

3 left

💡 Hint

Common Mistakes

Passing accuracy or labels instead of predictions.

Confusing scores with predicted labels.

The confusion matrix compares true labels with predicted labels, so predictions must be passed as the second argument.

4fill in blank

hard

Fill both blanks to create a dictionary of word counts from LLM output tokens.

Prompt Engineering / GenAI

word_counts = {word: [1] for word in tokens if word [2] stop_words}

Drag options to blanks, or click blank then click option'

Atokens.count(word)

Bin

Cnot in

Dlen(word)

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'in' instead of 'not in' to filter stop words.

Using len(word) instead of count for frequency.

We count how many times each word appears using tokens.count(word). We only include words that are not in the stop words list to focus on meaningful words.

5fill in blank

hard

Fill all three blanks to filter LLM predictions with confidence above threshold and map to labels.

Prompt Engineering / GenAI

filtered = [1]: label for label, score in zip(labels, scores) if score [2] threshold and label [3] valid_labels}

Drag options to blanks, or click blank then click option'

Alabel

B>

Cin

Dscore

Attempts:

3 left

💡 Hint

Common Mistakes

Using label as key instead of score.

Using '<' instead of '>' for filtering.

Checking label not in valid_labels instead of in.

We use score as the key to map labels. We filter scores greater than the threshold using >. We include only labels that are in the valid labels list.