Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement function to calculate Mean Reciprocal Rank metric #6065

Closed
Tracked by #6061
silvanocerza opened this issue Oct 16, 2023 · 0 comments · Fixed by #7042 or #7468
Closed
Tracked by #6061

Implement function to calculate Mean Reciprocal Rank metric #6065

silvanocerza opened this issue Oct 16, 2023 · 0 comments · Fixed by #7042 or #7468
Assignees
@silvanocerza
Labels
2.x Related to Haystack v2.0 P1 High priority, add to the next sprint topic:eval type:feature New feature or request

Comments

@silvanocerza
Copy link
Contributor

As specified in proposal #5794 we need to implement a function to calculate the Mean Reciprocal Rank metric.

Ideally the function should be part of the private interface and called only through the calculate_metrics function (see #6063). _calculate_mrr() could be a nice name.

For more detailed information check out the original proposal.
@silvanocerza silvanocerza added type:feature New feature or request 2.x Related to Haystack v2.0 labels Oct 16, 2023
@silvanocerza silvanocerza added this to the 2.0-beta milestone Oct 16, 2023
@silvanocerza silvanocerza mentioned this issue Oct 16, 2023
Closed
@Timoeller Timoeller removed this from the 2.0-beta milestone Oct 18, 2023
@masci masci added the P2 Medium priority, add to the next sprint if no P1 available label Dec 11, 2023
@silvanocerza silvanocerza self-assigned this Feb 16, 2024
@tholor tholor added P1 High priority, add to the next sprint and removed P2 Medium priority, add to the next sprint if no P1 available labels Feb 16, 2024
@awinml awinml mentioned this issue Feb 20, 2024
Merged
@silvanocerza silvanocerza reopened this Feb 21, 2024
@silvanocerza silvanocerza added P2 Medium priority, add to the next sprint if no P1 available and removed P1 High priority, add to the next sprint labels Feb 23, 2024
@silvanocerza silvanocerza added this to the 2.1.0 milestone Feb 23, 2024
@silvanocerza silvanocerza added P3 Low priority, leave it in the backlog and removed P2 Medium priority, add to the next sprint if no P1 available labels Feb 23, 2024
@masci masci added P1 High priority, add to the next sprint and removed P3 Low priority, leave it in the backlog labels Mar 15, 2024
@masci masci added P2 Medium priority, add to the next sprint if no P1 available and removed P1 High priority, add to the next sprint labels Mar 22, 2024
@mrm1001 mrm1001 removed this from the 2.1.0 milestone Mar 25, 2024
@masci masci added P3 Low priority, leave it in the backlog P1 High priority, add to the next sprint and removed P2 Medium priority, add to the next sprint if no P1 available P3 Low priority, leave it in the backlog labels Mar 28, 2024
@silvanocerza silvanocerza mentioned this issue Apr 4, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@silvanocerza silvanocerza

Labels
2.x Related to Haystack v2.0 P1 High priority, add to the next sprint topic:eval type:feature New feature or request
Projects
None yet
Milestone
No milestone
5 participants
@masci @tholor @Timoeller @silvanocerza @mrm1001