Evaluation of fine-tuned models in Prompt Engineering / GenAI - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to calculate the accuracy of a fine-tuned model's predictions.

Prompt Engineering / GenAI

accuracy = sum(predictions == [1]) / len(predictions)

Drag options to blanks, or click blank then click option'

Amodel

Btrue_labels

Cpredictions

Dinputs

Attempts:

3 left

💡 Hint

Common Mistakes

Comparing predictions to themselves instead of true labels.

Using the input data instead of labels.

Accuracy compares the model's predictions to the true labels to see how many are correct.

2fill in blank

medium

Complete the code to compute the loss of the fine-tuned model on test data.

Prompt Engineering / GenAI

loss = model.evaluate(test_inputs, [1])

Drag options to blanks, or click blank then click option'

Atrain_labels

Btest_inputs

Ctest_outputs

Dtrain_inputs

Attempts:

3 left

💡 Hint

Common Mistakes

Using training data instead of test data for evaluation.

Passing inputs instead of outputs as labels.

The model's loss is calculated by comparing predictions to the true test outputs (labels).

3fill in blank

hard

Fix the error in the code to generate predictions from the fine-tuned model.

Prompt Engineering / GenAI

predictions = model.[1](new_data)

Drag options to blanks, or click blank then click option'

Apredict

Btransform

Cevaluate

Dfit

Attempts:

3 left

💡 Hint

Common Mistakes

Using fit which trains the model instead of predicting.

Using evaluate which returns loss and metrics.

To get predictions, use the predict method on new data.

4fill in blank

hard

Fill both blanks to create a dictionary of accuracy and loss after evaluation.

Prompt Engineering / GenAI

results = {'accuracy': [1], 'loss': [2]

Drag options to blanks, or click blank then click option'

Aaccuracy_score(true_labels, predictions)

Bmodel.evaluate(test_inputs, test_labels)[0]

Cmodel.evaluate(test_inputs, test_labels)[1]

Daccuracy_score(predictions, true_labels)

Attempts:

3 left

💡 Hint

Common Mistakes

Swapping the order of loss and accuracy in the dictionary.

Using accuracy_score with arguments reversed.

Accuracy is computed by comparing true labels and predictions. Loss is the first value returned by model.evaluate.

5fill in blank

hard

Fill all three blanks to compute precision, recall, and F1 score for the fine-tuned model.

Prompt Engineering / GenAI

precision = precision_score(true_labels, [1])
recall = recall_score([2], predictions)
f1 = f1_score(true_labels, [3])

Drag options to blanks, or click blank then click option'

Apredictions

Btrue_labels

Dtest_labels

Attempts:

3 left

💡 Hint

Common Mistakes

Swapping true labels and predictions in metric functions.

Using test_labels instead of true_labels.

Precision and F1 score compare true labels to predictions. Recall also compares true labels to predictions.