-
-
Notifications
You must be signed in to change notification settings - Fork 283
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug]: Safety checker is enabled for LoRA training with no way to disable it #1352
Comments
Are you using txt2img or diffusers for image generation? If its diffusers it might be us but if it's txt2img its very likely automatic1111. Let me know either way. |
It makes absolutely no difference which generator I use for classifier images -- as soon I click Create Model button in Dreambooth tab, folders They are both referenced in I can generate NSFW images in AUTOMATIC1111 with the same EDIT: It seems that's coming from the diffusers package: https://github.com/search?q=repo%3Ahuggingface%2Fdiffusers+requires_safety_checker&type=code |
Got you. I don't think this should be happening unless the safety checker is actually specified. It seems to be fixed in dev already coz my model folders don't have it. Let me take a look. |
I replicated your problem and have a workaround but need to find a proper fix. We aren't enabling it in code so I suspect its from the default config. That config gets loaded with the model which enables it anywhere the pipeline is used that doesn't have it explicitly disabled. Rather than add a bunch of lines to disable it everywhere a pipeline could load it we can just remove it from the config completely. Ideally this should be at model creation but we can fix it in post for now. All you have to do is either delete the safety_checker section in model_index.json located in the working_dir of the model or set it's parameters to null. This is the section to delete, and how it appears nulled out. Yours will have the set to true and valid network names for it. "requires_safety_checker": null, Once it's been neutered you can safely delete the entire safety_checker folder. |
Thanks for looking into it. Yes, I figured out that workaround before creating an issue, or otherwise I wouldn't have been able to provide as much details as I did. That said, I am looking forward to the proper fix so I don't have to edit the file for every model I create because that's not the only problem — it also wastes bandwidth and makes useless disk writes (1.13 GB model.safetensors for safety_checker is added to every created model). On a side note, I think you should also properly disable Finally, it would be nice if someone looked at #1334 and tried to reproduce, I can't train extended LoRA anymore and I'd like to know what the cause might be. The only thing different from before is update to A1111 and Dreambooth. |
All I can say, even if you really need NSFW filters to opinionate morality of your code, it should at least yield a warning in a console! Or abort generation! Generating a BLACK rectangle is racist btw. |
Instead of manually changing this every time, is there a simple change to the code we can make to get it to work automatically? |
@saunderez Any ETA on a proper fix for this one? |
I just run: sed -i 's/requires_safety_checker": true/requires_safety_checker": null/' */working/model_index.json
sed -i 's/"stable_diffusion"/null/' */working/model_index.json
sed -i 's/"StableDiffusionSafetyChecker"/null/' */working/model_index.json in my Besides, I haven't been able to train one model ever with this tool, using it just to generate class images. In https://github.com/bmaltais/kohya_ss the safety checks are simply ignored by default and nobody cares. |
@paboum I don't have |
As I said, I am not interested in manual workarounds. This shouldn't be difficult to fix. |
This issue is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days |
Bump. |
This issue is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days |
@saunderez This is coming from So yes, it's coming from you using the defaults. It affects both classification image and sample image generation during training. |
Is there an existing issue for this?
What happened?
Yesterday I updated to the latest version of the extension and I tried training a LoRA on some NSFW content.
The class images generated were all black, and most of them weren't even being saved to the classifiers output folder.
I checked, and under the
model\dreambooth\MODEL_NAME\working
there was asafety_checker
folder with 1.13 GBmodel.safetensors
file andconfig.json
with commit hashcb41f3a270d63d454d385fc2e4f571c487c253c5
and name/pathCompVis/stable-diffusion-safety-checker
.Also, I noticed that
model_index.json
in theworking
folder now contains:Before in my older LoRA trainings (from the same base model) it looked like this:
Steps to reproduce the problem
Commit and libraries
Initializing Dreambooth
Dreambooth revision: cf086c5
Successfully installed fastapi-0.94.1
[+] xformers version 0.0.20 installed.
[+] torch version 2.0.1+cu118 installed.
[+] torchvision version 0.15.2+cu118 installed.
[+] accelerate version 0.21.0 installed.
[+] diffusers version 0.19.3 installed.
[+] transformers version 4.30.2 installed.
[+] bitsandbytes version 0.35.4 installed.
Command Line Arguments
Console logs
Not relevant because there is no error, it's just undesirable behavior.
Additional information
No response
The text was updated successfully, but these errors were encountered: