-
Notifications
You must be signed in to change notification settings - Fork 26.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix RoPE config validation for FalconConfig + various config typos #26929
Conversation
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for fixing my sloppy english :D
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the nits
…uggingface#26929) * Resolve incorrect ValueError in RoPE config for Falcon * Add broken codeblock tag in Falcon Config * Fix typo: an float -> a float * Implement copy functionality for Fuyu and Persimmon for RoPE scaling validation * Make style
…uggingface#26929) * Resolve incorrect ValueError in RoPE config for Falcon * Add broken codeblock tag in Falcon Config * Fix typo: an float -> a float * Implement copy functionality for Fuyu and Persimmon for RoPE scaling validation * Make style
…uggingface#26929) * Resolve incorrect ValueError in RoPE config for Falcon * Add broken codeblock tag in Falcon Config * Fix typo: an float -> a float * Implement copy functionality for Fuyu and Persimmon for RoPE scaling validation * Make style
What does this PR do?
One change that fixes a bug:
config = FalconConfig(alibi=False, rope_scaling={"type": "linear", "factor": 2.0})
would fail. (1fbd699)And three changes that fix typos/improve maintainability etc.
I can split this up into two smaller PRs if preferred - just let me know.
Details
Fix RoPE config validation
Running this on main produces:
Which is obviously nonsensical. This is caused by
if self.rotary: raise ValueError
whereself.rotary
is justnot self.alibi
. So, this ValueError is thrown when alibi isFalse
, not when it isTrue
.The other 3 changes speak for themselves.
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
No new tests - I can add one for the first bug fix if preferred.
Who can review?
@gante