Implement function to calculate metrics from EvaluationResult
#6063
Labels
2.x
Related to Haystack v2.0
P2
Medium priority, add to the next sprint if no P1 available
type:feature
New feature or request
As specified in proposal #5794 we need to implement the
calculate_metrics()
function.The
calculated_metrics()
function will expect this arguments:evaluation_result: EvaluationResult
metric: Union[Metric, MetricCalculator]
**kwargs
And will return an instance of
MetricsResult
.Metric
can be either a simpleEnum
or aLiteral
. Probably anEnum
will ease the discoverability of supported metrics.MetricCalculator
will be aCallable[..., MetricResult]
.**kwargs
will be passed to the function used to calculate the metrics, can be obviously omitted.For more detailed information check out the original proposal.
The text was updated successfully, but these errors were encountered: