Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement statistical-based evaluation and metrics calculation #6061

Closed
19 tasks done
Tracked by #6672
silvanocerza opened this issue Oct 16, 2023 · 1 comment
Closed
19 tasks done
Tracked by #6672

Implement statistical-based evaluation and metrics calculation #6061

silvanocerza opened this issue Oct 16, 2023 · 1 comment
Assignees
Labels
2.x Related to Haystack v2.0 epic topic:eval type:feature New feature or request
Milestone

Comments

@silvanocerza
Copy link
Contributor

silvanocerza commented Oct 16, 2023

As outlined in proposal #5794 we need to implement evaluation and metrics calculation for Haystack 2.x.

To complete this epic we need to close the following issues:

Tasks

  1. 2.x type:feature
    silvanocerza
  2. 2.x type:feature
    silvanocerza
  3. 2.x P2 type:feature
  4. 2.x P2 type:feature
  5. 2.x P2 type:feature
  6. 2.x P2 type:feature
  7. 2.x P2 type:feature
  8. 2.x P1 type:feature
    silvanocerza
  9. 2.x P1 topic:eval type:feature
    silvanocerza
  10. 2.x P1 topic:eval type:feature
    silvanocerza
  11. 2.x P1 topic:eval type:feature
    silvanocerza
  12. 2.x P1 type:feature wontfix
    silvanocerza
  13. 2.x P1 topic:eval type:feature
    silvanocerza
  14. P1 topic:eval type:documentation
    masci
  15. P1 topic:eval type:documentation
    julian-risch
  16. P1 topic:eval type:documentation
    julian-risch
  17. P1 topic:eval type:documentation
    julian-risch
  18. P1 topic:eval type:documentation
    julian-risch
  19. P1 topic:eval type:documentation
    julian-risch
@silvanocerza silvanocerza added type:feature New feature or request epic 2.x Related to Haystack v2.0 labels Oct 16, 2023
@silvanocerza silvanocerza added this to the 2.0-beta milestone Oct 16, 2023
@Timoeller Timoeller removed this from the 2.0-beta milestone Oct 16, 2023
@awinml
Copy link
Contributor

awinml commented Dec 1, 2023

@silvanocerza I have opened a draft PR (#6464) that adds Evaluation and Metrics calculation for components and Pipelines. Requesting comments on the overall approach. Thanks.

@silvanocerza silvanocerza self-assigned this Feb 16, 2024
@silvanocerza silvanocerza added this to the 2.1.0 milestone Feb 23, 2024
@masci masci changed the title Implement evaluation and metrics calculation Implement statistical-based evaluation and metrics calculation Apr 7, 2024
@masci masci closed this as completed Apr 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.x Related to Haystack v2.0 epic topic:eval type:feature New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants