Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --metric/-m option to eval command to select metric(s) #558

Merged
merged 1 commit into from
Jan 27, 2022

Conversation

osma
Copy link
Member

@osma osma commented Jan 27, 2022

This PR implements a new option -m/--metric for the annif eval command. It can be used (and repeated) to select the metric(s) to calculate, instead of all of them. Example:

annif eval my-project -m F1@5 -m NDCG /path/to/my-corpus

Fixes #545

This is useful for DVC as well to avoid introducing a lot of unnecessary metrics into experiments.

@sonarcloud
Copy link

sonarcloud bot commented Jan 27, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@codecov
Copy link

codecov bot commented Jan 27, 2022

Codecov Report

Merging #558 (8da9e49) into master (49aa5df) will increase coverage by 0.03%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #558      +/-   ##
==========================================
+ Coverage   99.46%   99.50%   +0.03%     
==========================================
  Files          82       82              
  Lines        5428     5443      +15     
==========================================
+ Hits         5399     5416      +17     
+ Misses         29       27       -2     
Impacted Files Coverage Δ
annif/cli.py 99.63% <100.00%> (+<0.01%) ⬆️
annif/eval.py 100.00% <100.00%> (ø)
tests/test_cli.py 100.00% <100.00%> (ø)
annif/backend/yake.py 98.23% <0.00%> (+1.76%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 49aa5df...8da9e49. Read the comment docs.

@osma osma added this to the 0.56 milestone Jan 27, 2022
@osma osma merged commit 718bee3 into master Jan 27, 2022
@osma osma deleted the issue545-eval-metric-option branch January 27, 2022 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Select metrics for eval command using an option
1 participant