-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add inpainting pipeline to diffuser conversion #356
Comments
If we want to implement an inpainting model type for the extras file, https://github.com/ssube/onnx-web/blob/main/api/onnx_web/convert/__main__.py will also need to be changed around line 339 |
I have some code for this on the same branch as tiled VAE. If I can get that working today, I'll merge both, otherwise I will cherry-pick this fix. |
I added support for the inpaint pipeline to the
I need to figure out if that should be fixed in place or if #337 will fix this. |
The
The stock SD v1.5 and v2.1 inpainting models convert correctly. This one appears to have a different shape, but that same shape error appears if you use one of the successfully-converted inpainting models with a non-inpainting pipeline, so it's not totally new. I need to compare the tensors in the models and see what is actually different here. It may still be a bug in the ckpt loader. |
Per huggingface/diffusers#3254 (comment), It seems like the most efficient way to do this will be to create the new |
Currently trying to add an inpainting model results in a size mistmatch error during conversion. Desired bhavior is to convert the inpainting model.
Problem appears to be in https://github.com/ssube/onnx-web/blob/main/api/onnx_web/convert/diffusion/diffusers.py#L3
line 214, there should be an alternate pipeline (https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_inpaint.py) for inpainting diffusers, and around line 421 as well (https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_inpaint.py)
The text was updated successfully, but these errors were encountered: