[docs] Convert weights #345
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
docs/source/convert-weights.md
Outdated
| ```py | ||
| from transformers import AutoModel | ||
|
|
||
| model = AutoModel.from_pretrained( | ||
| "my-safe-model", revision="refs/pr/1", use_safetensors=True | ||
| ) | ||
| ``` | ||
|
|
||
| Another way to convert your `.bin` files is to use the [`~safetensors.torch.save_model`] function: | ||
|
|
||
| ```py | ||
| from transformers import AutoModel | ||
| from safetensors.torch import save_model | ||
|
|
||
| unsafe_model = AutoModel.from_pretrained("my-unsafe-model") | ||
| save_model(unsafe_model, "model.safetensors") | ||
| ``` |
There was a problem hiding this comment.
| ```py | |
| from transformers import AutoModel | |
| model = AutoModel.from_pretrained( | |
| "my-safe-model", revision="refs/pr/1", use_safetensors=True | |
| ) | |
| ``` | |
| Another way to convert your `.bin` files is to use the [`~safetensors.torch.save_model`] function: | |
| ```py | |
| from transformers import AutoModel | |
| from safetensors.torch import save_model | |
| unsafe_model = AutoModel.from_pretrained("my-unsafe-model") | |
| save_model(unsafe_model, "model.safetensors") | |
| ``` |
I'm not a big fan of suggesting either way here.
- AutoModel will miss the model's head leading to bad models being saved and we cannot invent the correct head for users. Also it requires
transformerswhich safetensors doesn't require. - This should work and is the preferred way for doing it when not using transformers, however, it might cause issues with transformers models because of weight naming (some weights share the same name, safetensors will discard any and everything will work fine, but that might be breaking the names transformers expects to see and clash with their own internals).
Using the space should be the go-to IMO.
There was a problem hiding this comment.
bindings/python/convert.py is basically the code run on the space too.
There was a problem hiding this comment.
The space has been crashing/not working for a number of users, which is why I requested a small guide (you can check this discussion thread).
There was a problem hiding this comment.
I could remove the logit sanity check, it should work for more models. Still going to be dead slow..
If we could at least add progressbars that'd help but I'm not sure how to actually do that.
There was a problem hiding this comment.
@christophe-rannou @co42 Any way we could bump/replicate this space to have more replicas ?
It's chugging pretty much everything it can atm
There was a problem hiding this comment.
This should work and is the preferred way for doing it when not using transformers, however, it might cause issues with transformers models because of weight naming (some weights share the same name, safetensors will discard any and everything will work fine, but that might be breaking the names transformers expects to see and clash with their own internals).
Maybe we can add a warning here to let users know they might run into issues if they use the save_model method with transformers and they can try it on their own if they want?
Otherwise, we can just keep the Convert Space option and have a warning that says it may be slower for larger models + point them towards bindings/python/convert.py if they want to convert it locally.
From internal Slack thread, this PR adds a short guide for converting weights to
.safetensors. Let me know if I'm missing anything!