-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Don't use built-in VAE #4
Comments
Changing VAEs will not change the watermark on SDXL. |
Hello @IdiotSandwichTheThird and @AshtakaOOf , we are aware of the opinions of XL’s VAE is watermarked by some methods. Nevertheless, we do not have enough evidences to support this. We do not know any existing research or methods that can watermark a latent diffusion’s first stage model. The artifacts in your images are probably caused by the distribution of training data of XL’s vae. The XL’s vae is (likely to be) trained with many images with crisp and sharp texts in it so that its performance can be better for encoding/decoding texts, and the downside is probably some of your mentioned “chromatic aberration” because those training images have many “subpixels” for crisp and sharp texts. Feel free to take a look at “subpixels” https://en.wikipedia.org/wiki/Subpixel_rendering to understand the “chromatic aberration” and text rendering, and this probably explains XL’s vae behaviors. Finally, if anyone has stronger evidences to support the claim of “vae watermark”, please let us know. |
The VAE issue with "watermarking" was an error that was fixed 2 days after the SDXL 1.0 release, if you download the model or VAE from their repository after that time then there will be no "watermarking" effect as the model's baked VAE has been reverted to the .9 version and re-uploaded. |
@lllyasviel Stability AI released official SDXL 1.0 with VAE from 0.9 to solve artifacts problems in their original repo (sd_xl_base_1.0_0.9vae.safetensors and sd_xl_refiner_1.0_0.9vae.safetensors) - you can check out discussion in diffusers issue #4310, or just compare some images from original, and fixed release by yourself. By default I'd suggest to go with updated versions (those with VAE from 0.9). I've locally replaced paths the model files (in
Results - output from original SDXL 1.0: and from models with fixed VAE - artifacts near mirror are gone: |
This looks more convincing. We will have some actions on this. |
The built in VAE is watermarked and degrades image quality, leading to noticeable aliasing and chromatic aberration like effects, as seen in this example https://i.imgur.com/SCY5P5z.png .
The text was updated successfully, but these errors were encountered: