Make using safetensors files automated. #27571
Conversation
If `use_safetensors=True` is used, and it doesn't exist: - Don't crash just yet - Lookup for an open PR containing it. - If yes, use that instead - If not, touch the space to convert, wait for conversion to be finished and the PR to be opened - Use that new PR - Profit.
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
There was a problem hiding this comment.
First of all it's very cool, thanks for working on it! It gets the job done. I left mostly random nits, but here's the gist of what I think:
In terms of the implementation/what can be accepted in transformers:
- the specific module looks good to me.
- I would relax the
websocketsrequirement in favor ofrequests
General workflow
- Right now it asks for a token and opens a PR under the token's owner (https://huggingface.co/lysandre/ctrl-clone-2/discussions/1). I'd much favor if it was opened from the bot's perspective. Having the PR be opened by a third party makes me uneasy as you have no certainty it was actually opened through this workflow.
- I think we might have an issue with sharded checkpoints, will need to check
| def previous_pr(api: "HfApi", model_id: str, pr_title: str) -> Optional["Discussion"]: | ||
| try: | ||
| main_commit = api.list_repo_commits(model_id)[0].commit_id | ||
| discussions = api.get_repo_discussions(repo_id=model_id) | ||
| except Exception: | ||
| return None | ||
| for discussion in discussions: | ||
| if discussion.status == "open" and discussion.is_pull_request and discussion.title == pr_title: | ||
| commits = api.list_repo_commits(model_id, revision=discussion.git_reference) | ||
|
|
||
| if main_commit == commits[1].commit_id: | ||
| return discussion | ||
| return None |
There was a problem hiding this comment.
we can implem a Hub API to replace this whole function IMO cc @SBrandeis
There was a problem hiding this comment.
Courtesy of @SBrandeis (huggingface/huggingface_hub#1845):
for discussion in get_repo_discussions(
repo_id="openai/whisper-large-v3",
author="sanchit-gandhi",
discussion_type="pull_request",
discussion_status="open",
):
...There was a problem hiding this comment.
Thanks Simon and Lucain! Applied in d32a18e
* Websocket -> SSE * Support sharded + tests +cleanup a * env var
Co-authored-by: Wauplin <[email protected]>
LysandreJik
changed the title
|
It's starting to look good! I've tried it with Intel/neural-chat-7b-v3-1, a 7b model, and it took ~3 minutes between when I sent the initial request and when it started downloading the safetensors checkpoint. The first minute was spent waiting for previous conversions to wrap up. Is there a possibility for us to speed this up even more/parallelize it? |
|
@ArthurZucker can you take a look when you have a second? |
|
Reviewing now |
Co-authored-by: ArthurZucker <[email protected]>
|
Thanks for the reviews, merging! |
If
use_safetensors=Trueis used, and it doesn't exist:and the PR to be opened
What does this PR do?
Fixes # (issue)
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.