LangSmith evaluators in LangChain - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to import the Evaluator class from LangSmith.

LangChain

from langsmith.evaluation import [1]

Drag options to blanks, or click blank then click option'

AEvaluator

BEvaluatorClient

CEvaluation

DEval

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'EvaluatorClient' which does not exist.

Importing 'Evaluation' instead of 'Evaluator'.

The Evaluator class is imported from langsmith.evaluation to create evaluators.

2fill in blank

medium

Complete the code to create an Evaluator instance with the name 'my_eval'.

LangChain

evaluator = Evaluator(name=[1])

Drag options to blanks, or click blank then click option'

A'my_eval'

B'default'

C'eval1'

D'test'

Attempts:

3 left

💡 Hint

Common Mistakes

Using names other than 'my_eval'.

Forgetting quotes around the string.

The Evaluator instance should be named 'my_eval' as specified.

3fill in blank

hard

Fix the error in the code to evaluate a prediction with the evaluator.

LangChain

result = evaluator.evaluate(prediction=[1])

Drag options to blanks, or click blank then click option'

ANone

B123

CTrue

D'This is a prediction'

Attempts:

3 left

💡 Hint

Common Mistakes

Passing a number instead of a string.

Passing None or boolean values.

The prediction argument expects a string representing the predicted output.

4fill in blank

hard

Fill both blanks to create an Evaluator and evaluate a prediction with input text.

LangChain

evaluator = Evaluator(name=[1])
result = evaluator.evaluate(prediction=[2])

Drag options to blanks, or click blank then click option'

A'eval_test'

B'Sample prediction'

C'Wrong prediction'

D'test_eval'

Attempts:

3 left

💡 Hint

Common Mistakes

Mixing up the evaluator name and prediction strings.

Using invalid strings for either blank.

The evaluator is named 'eval_test' and the prediction is 'Sample prediction'.

5fill in blank

hard

Fill all three blanks to create an Evaluator, evaluate a prediction, and print the result.

LangChain

evaluator = Evaluator(name=[1])
pred = [2]
result = evaluator.evaluate(prediction=pred)
print([3])

Drag options to blanks, or click blank then click option'

A'final_eval'

B'Final prediction'

Cresult

Dpred

Attempts:

3 left

💡 Hint

Common Mistakes

Printing the prediction instead of the result.

Using incorrect variable names.

Forgetting quotes around strings.

The evaluator is named 'final_eval', the prediction string is 'Final prediction', and the printed output is the evaluation result.