Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement function to calculate Semantic Answer Similarity metric #6069

Closed
Tracked by #6061
silvanocerza opened this issue Oct 16, 2023 · 2 comments · Fixed by #6877 or #7428
Closed
Tracked by #6061

Implement function to calculate Semantic Answer Similarity metric #6069

silvanocerza opened this issue Oct 16, 2023 · 2 comments · Fixed by #6877 or #7428
Assignees
@silvanocerza
Labels
2.x Related to Haystack v2.0 P1 High priority, add to the next sprint topic:eval type:feature New feature or request
Milestone
2.1.0

Comments

@silvanocerza
Copy link
Contributor

As specified in proposal #5794 we need to implement a function to calculate the Semantic Answer Similarity metric.

Ideally the function should be part of the private interface and called only through the calculate_metrics function (see #6063). _calculate_sas() could be a nice name.

For more detailed information check out the original proposal.
@silvanocerza silvanocerza added type:feature New feature or request 2.x Related to Haystack v2.0 labels Oct 16, 2023
@silvanocerza silvanocerza added this to the 2.0-beta milestone Oct 16, 2023
@silvanocerza silvanocerza mentioned this issue Oct 16, 2023
Closed
@Timoeller Timoeller removed this from the 2.0-beta milestone Oct 18, 2023
@masci masci added the P2 Medium priority, add to the next sprint if no P1 available label Dec 11, 2023
@awinml awinml mentioned this issue Jan 31, 2024
Merged
@silvanocerza silvanocerza reopened this Feb 21, 2024
@silvanocerza silvanocerza added this to the 2.1.0 milestone Feb 23, 2024
@silvanocerza silvanocerza added P3 Low priority, leave it in the backlog and removed P2 Medium priority, add to the next sprint if no P1 available labels Feb 23, 2024
@mrm1001 mrm1001 mentioned this issue Mar 22, 2024
Closed
@masci masci added P1 High priority, add to the next sprint and removed P3 Low priority, leave it in the backlog labels Mar 22, 2024
@silvanocerza silvanocerza mentioned this issue Mar 26, 2024
Merged
@davidsbatista
Copy link
Contributor

are there more details on the specific Semantic Answer Similarity metric that will be used?

@davidsbatista
Copy link
Contributor

found it: #7428

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@silvanocerza silvanocerza

Labels
2.x Related to Haystack v2.0 P1 High priority, add to the next sprint topic:eval type:feature New feature or request
Projects
None yet
Milestone
2.1.0
5 participants
@masci @Timoeller @silvanocerza @mrm1001 @davidsbatista