Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

detect Textual Inversion type or get it from the prompt token #262

Closed
ssube opened this issue Mar 19, 2023 · 1 comment
Closed

detect Textual Inversion type or get it from the prompt token #262

ssube opened this issue Mar 19, 2023 · 1 comment
Labels
model/inversion status/fixed issues that have been fixed and released type/bug broken features
Milestone

Comments

@ssube
Copy link
Owner

ssube commented Mar 19, 2023

The code for Textual Inversions cannot currently detect whether it's an sd-concept from the HF hub or a tensor file with embeddings. The blend code is not fully integrated with the model fetch code, and does its own fetch from HF hub.

Inversions used with the <inversion:name:1.0> token will always be loaded as embeddings, with no way to use a concept-style inversion by token.

@ssube ssube added status/planned issues that have been planned but not started type/bug broken features labels Mar 19, 2023
@ssube ssube added this to the v0.9 milestone Mar 19, 2023
@ssube
Copy link
Owner Author

ssube commented Mar 20, 2023

The HF concepts do not really need a second file and contain their own token as the only key in the tensor, making it pretty easy to detect them once the inversion code was set up to use the regular model fetch code.

Using an HTTPS or local path source for an inversion will behave normally, but huggingface:// paths will only download the learned_embeds.bin file, like https://huggingface.co/sd-concepts-library/concept-art/blob/main/learned_embeds.bin

[2023-03-20 01:35:23,724] DEBUG: 349112 140247140089856 onnx_web.convert.diffusion.textual_inversion: detected Textual Inversion concept: ['<concept-art>']                               
[2023-03-20 01:35:23,724] DEBUG: 349112 140247140089856 onnx_web.convert.diffusion.textual_inversion: found embeddings for 2 tokens: ['concept-art', '<concept-art>']                     
[2023-03-20 01:35:23,918] INFO: 349112 140247140089856 onnx_web.convert.diffusion.textual_inversion: blending Textual Inversion ../models/inversion/cloud with weight of 0.25 for token cloud    

@ssube ssube added status/fixed issues that have been fixed and released and removed status/planned issues that have been planned but not started labels Mar 20, 2023
@ssube ssube closed this as completed Mar 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
model/inversion status/fixed issues that have been fixed and released type/bug broken features
Projects
None yet
Development

No branches or pull requests

1 participant