Langsmith Evaluation custom metrics #26639
Unanswered
HemanthVikash
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I am writing an evaluation that runs for n=5 iterations in each example and I want to see what the output scores are. One such score that I am evaluating is the data_row_count. For each example, I can see the averaged data_row_count on langsmith. However, there is seemingly no way to calculate variance or standard deviation. Is there an easy way to currently do that?
Here is my eval code:
Here is my evaluator:
Additionally here is a screenshot of the langsmith eval:
In this screenshot I essentially want to have different metrics (variance or SD) across the runs for each value captured (data_row_count, data_column_count, sql_executable)
Beta Was this translation helpful? Give feedback.
All reactions