Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: fix for R^2 and SMAPE #2744

Closed
profPlum opened this issue Sep 15, 2024 · 2 comments · Fixed by #2771
Closed

Docs: fix for R^2 and SMAPE #2744

profPlum opened this issue Sep 15, 2024 · 2 comments · Fixed by #2771
Labels
documentation Improvements or additions to documentation good first issue Good for newcomers

Comments

@profPlum
Copy link

profPlum commented Sep 15, 2024

📚 Documentation

SMAPE docs say: "smape [...] A tensor with non-negative floating point smape value between 0 and 1"
R^2 docs say: "By definition the score is bounded between 0 and 1"

Both of these are incorrect:

  • SMAPE ranges between [0,2]
  • R^2 ranges between (-∞,1]
    • This is true when generalized beyond linear models, as is the case for torchmetrics.
    • But tbh it might be helpful to add an optional flag that can clip the range to [-1,1], for the sake of tensorboard readability? I'll leave this decision to you.

P.S.
Here is an example tensorboard plot that is hard to read due to exploding negative R^2:
Screenshot 2024-09-15 at 2 10 17 PM

@profPlum profPlum added the documentation Improvements or additions to documentation label Sep 15, 2024
@Borda
Copy link
Member

Borda commented Sep 16, 2024

Thank you @profPlum mind sending a PR to fix this issue? 🐰
cc: @SkafteNicki

@Borda Borda added the good first issue Good for newcomers label Sep 16, 2024
@Borda Borda changed the title Please fix docs for R^2 and SMAPE docs Docs: fix for R^2 and SMAPE Sep 16, 2024
@SkafteNicki
Copy link
Member

PR #2771 fixes the incorrect statements in the docs but the clipping feature is not implemented as it is a very specific feature that depends on the usecase (sometimes it may be beneficial to know if the r2 score is highly negative indicating a very bad model)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants