Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LORA - Some stopped working with new updates. #938

Closed
scsp85 opened this issue Nov 13, 2023 · 10 comments
Closed

LORA - Some stopped working with new updates. #938

scsp85 opened this issue Nov 13, 2023 · 10 comments

Comments

@scsp85
Copy link

scsp85 commented Nov 13, 2023

Some LORA stopped working with updates this weekend. I noticed that some Lora will not load, without throwing an error, just manifesting without any impact to the image. I’m running the newest update as of this morning, on a Windows 11 machine with a GTX 3090, intel Core i7, and 24 GB of ram.

Below I load a “Kelly” LORA that is homemade, Open Trainer, network rank 64, FP16, and it works.

The next is a newly trained “JakkiSDXLRP” LORA is 256 rank and FP16, and it doesn’t work or load in fooocus, yet it does work in Automatic 1111.

The last is an older “Jakki_SDXL” LORA, 128 network rank, FP32, that 100% was working before the updates this weekend, and now also will no load. It also works in Automatic 1111 as well.

Console:

[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 3.0
[Parameters] Seed = 7871279234602423560
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
model_type EPS
adm 2816
Using xformers attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using xformers attention in VAE
extra keys {'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_l.text_projection'}
Base model loaded: A:\Fooocus\Fooocus\models\checkpoints\realisticStockPhoto_v10.safetensors
Request to load LoRAs [('SDXL_FILM_PHOTOGRAPHY_STYLE_BetaV0.4.safetensors', 0.25), ('kelly.safetensors', 0.25), ('None', 0.25), ('None', 0.25), ('None', 0.25)] for model [A:\Fooocus\Fooocus\models\checkpoints\realisticStockPhoto_v10.safetensors].
Loaded LoRA [A:\Fooocus\Fooocus\models\loras\SDXL_FILM_PHOTOGRAPHY_STYLE_BetaV0.4.safetensors] for model [A:\Fooocus\Fooocus\models\checkpoints\realisticStockPhoto_v10.safetensors] with 1052 keys at weight 0.25.
Loaded LoRA [A:\Fooocus\Fooocus\models\loras\kelly.safetensors] for model [A:\Fooocus\Fooocus\models\checkpoints\realisticStockPhoto_v10.safetensors] with 722 keys at weight 0.25.
Requested to load SDXLClipModel
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 1.13 seconds
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] test photo, delicate, beautiful, intricate, epic composition, elegant, dynamic, vibrant colors, highly detailed, dramatic light, sharp focus, cinematic, confident, new, shiny, colorful, color, advanced, romantic, futuristic, magic, vivid, fine detail, elaborate, complex, glowing, magical, pure, amazing, awesome, symmetry, flowing, background, illuminated
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] test photo, sunny, dynamic, composition, dramatic, cinematic, detailed, exposed, full, shining, light, real, shiny, vibrant, beautiful, symmetry, perfect, focused, creative, positive, unique, attractive, cute, elegant, best, pure, fine detail, very clear, color, pristine, joyful, awesome, artistic, thought, complex background, professional
[Fooocus] Encoding positive #1 ...
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
Preparation time: 69.54 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.02916753850877285, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 8.25 seconds
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:09<00:00, 3.05it/s]
Image generated with private log at: A:\Fooocus\Fooocus\outputs\2023-11-13\log.html
Generating and saving time: 19.22 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.02916753850877285, sigma_max = 14.614643096923828
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:09<00:00, 3.08it/s]
Image generated with private log at: A:\Fooocus\Fooocus\outputs\2023-11-13\log.html
Generating and saving time: 10.76 seconds
Total time: 103.97 seconds
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 3.0
[Parameters] Seed = 4233827007911924040
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
Request to load LoRAs [('SDXL_FILM_PHOTOGRAPHY_STYLE_BetaV0.4.safetensors', 0.25), ('jakkiSDXLRP.safetensors', 0.25), ('None', 0.25), ('None', 0.25), ('None', 0.25)] for model [A:\Fooocus\Fooocus\models\checkpoints\realisticStockPhoto_v10.safetensors].
Loaded LoRA [A:\Fooocus\Fooocus\models\loras\SDXL_FILM_PHOTOGRAPHY_STYLE_BetaV0.4.safetensors] for model [A:\Fooocus\Fooocus\models\checkpoints\realisticStockPhoto_v10.safetensors] with 1052 keys at weight 0.25.
Requested to load SDXLClipModel
Loading 1 new model
unload clone 1
[Fooocus Model Management] Moving model(s) has taken 0.71 seconds
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] test photo, cinematic, professional composition, dramatic atmosphere, detailed, vibrant colors, ambient light, dynamic, sharp focus, cool color, open background, revealing scenic full bright glorious perfect, deep aesthetic, inspired, rich vivid intricate detail, creative, positive energetic, joyful, unique, glowing, pure, beautiful, focused, amazing, romantic, exciting, cute, surreal, symmetry
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] test photo, dramatic light, cinematic background, sharp, quality, highly detailed, intricate, beautiful, elegant, dynamic, vibrant colors, perfect, focus, epic, professional, winning, unique, best, creative, positive, cute, friendly, cheerful, pretty, confident, smart, color, healthy, bright, inspirational, attractive, charming, nice, determined, passionate
[Fooocus] Encoding positive #1 ...
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
Preparation time: 290.91 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.02916753850877285, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
unload clone 2
[Fooocus Model Management] Moving model(s) has taken 1.68 seconds
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:09<00:00, 3.09it/s]
Image generated with private log at: A:\Fooocus\Fooocus\outputs\2023-11-13\log.html
Generating and saving time: 12.42 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.02916753850877285, sigma_max = 14.614643096923828
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:09<00:00, 3.09it/s]
Image generated with private log at: A:\Fooocus\Fooocus\outputs\2023-11-13\log.html
Generating and saving time: 10.67 seconds
Total time: 318.04 seconds
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 3.0
[Parameters] Seed = 12692661723234709
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
Request to load LoRAs [('SDXL_FILM_PHOTOGRAPHY_STYLE_BetaV0.4.safetensors', 0.25), ('jakki_SDXL1.safetensors', 0.25), ('None', 0.25), ('None', 0.25), ('None', 0.25)] for model [A:\Fooocus\Fooocus\models\checkpoints\realisticStockPhoto_v10.safetensors].
Loaded LoRA [A:\Fooocus\Fooocus\models\loras\SDXL_FILM_PHOTOGRAPHY_STYLE_BetaV0.4.safetensors] for model [A:\Fooocus\Fooocus\models\checkpoints\realisticStockPhoto_v10.safetensors] with 1052 keys at weight 0.25.
Requested to load SDXLClipModel
Loading 1 new model
unload clone 1
[Fooocus Model Management] Moving model(s) has taken 0.69 seconds
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] test photo, gorgeous, intricate, elegant, breathtaking, light, shining, highly detailed, incredible quality, thought, cinematic, dramatic ambient, epic, focus, coherent, great composition, dynamic, vibrant colors, perfect, beautiful, artistic, complex, color, fine detail, professional, best, clear, trendy, futuristic, cool, awesome, creative, positive, cute
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] test photo, extremely detailed, sharp focus, elegant, dynamic dramatic cinematic perfect, fine detail, colorful, beautiful, stunning, symmetry, clear background, vivid color, highly saturated colors, inspired, intricate, very inspirational, friendly, illustrious, sweet, pretty, open, radiant, professional, best, fair, hopeful, thought, magic, epic, magical, gorgeous, astounding
[Fooocus] Encoding positive #1 ...
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
Preparation time: 7.62 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.02916753850877285, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
unload clone 2
[Fooocus Model Management] Moving model(s) has taken 1.46 seconds
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:09<00:00, 3.10it/s]
Image generated with private log at: A:\Fooocus\Fooocus\outputs\2023-11-13\log.html
Generating and saving time: 12.11 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.02916753850877285, sigma_max = 14.614643096923828
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:09<00:00, 3.08it/s]
Image generated with private log at: A:\Fooocus\Fooocus\outputs\2023-11-13\log.html
Generating and saving time: 10.73 seconds
Total time: 34.41 seconds

@lllyasviel
Copy link
Owner

upload the lora with problem

@scsp85
Copy link
Author

scsp85 commented Nov 13, 2023

upload the lora with problem

These are people’s likenesses, so I’m not comfortable uploading them online. I can attempt to use the same settings to train a new LORA on a different concept, and test.

I also plan on a clean install later today, and I’ll report back.

@scsp85
Copy link
Author

scsp85 commented Nov 13, 2023

upload the lora with problem

These are people’s likenesses, so I’m not comfortable uploading them online. I can attempt to use the same settings to train a new LORA on a different concept, and test.

I also plan on a clean install later today, and I’ll report back.

Clean install at lunch, no improvement. Also tested resizing with network rank 64 in Kohya_SS, still will not load. Will try to create another LORA this evening to upload.

@scsp85
Copy link
Author

scsp85 commented Nov 13, 2023

Does it matter that I trained both the text encoder and the Unet on the two LORA that are not working, and only the Unet on the functioning one?

@scsp85
Copy link
Author

scsp85 commented Nov 14, 2023

It appears that if you train the Lora's text encoders it will not load. Here are two example Lora, the one with the encoder trained and Unet will not load. The one with jus the Unet trained will load. They aren't very well trained, so don't expect a beautiful rendition of my dog, but just 13 images for 100 epoch.

GeorgiaDog Lora

Console:

[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 2588447414007137651
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
Request to load LoRAs [('sd_xl_offset_example-lora_1.0.safetensors', 0.1), ('GeorgiaDogECUN_R.safetensors', 0.98), ('None', 0.1), ('None', 0.1), ('None', 0.1)] for model [Z:\Fooocus\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors].


Requested to load SDXLClipModel
Loading 1 new model
unload clone 1
[Fooocus Model Management] Moving model(s) has taken 1.65 seconds
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] GeorgiaDog is laying in the grass by the pond, bright, inspired, highly detailed, intricate, sharp focus, cinematic light, beautiful, classic, designed, vivid colors, open background, symmetry, contemporary, fresh, fantastic, creative, positive, unique, pure, attractive, elegant, perfect, cute, very coherent, artistic, color, illuminated, outstanding, epic
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] GeorgiaDog is laying in the grass by the pond, bright, inspired color, intricate detail, cinematic, clear, crisp, sharp, rich, gorgeous, stunning, highly detailed, romantic, designed, vivid, beautiful, infinite, symmetry, light, atmosphere, perfect, ambient, magic, pretty, scenic, thought, professional, best, artistic, winning, new, novel
[Fooocus] Encoding positive #1 ...
[Fooocus Model Management] Moving model(s) has taken 0.12 seconds
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
Preparation time: 10.75 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.02916753850877285, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 5.12 seconds
100%|███████████████████████████████████████████████████████████████████████████████| 30/30 [00:25<00:00, 1.16it/s]
Image generated with private log at: Z:\Fooocus\outputs\2023-11-13\log.html
Generating and saving time: 32.85 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.02916753850877285, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 3.43 seconds
100%|███████████████████████████████████████████████████████████████████████████████| 30/30 [00:26<00:00, 1.15it/s]
Image generated with private log at: Z:\Fooocus\outputs\2023-11-13\log.html
Generating and saving time: 31.12 seconds
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
[Fooocus Model Management] Moving model(s) has taken 0.71 seconds
Total time: 78.57 seconds
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 2588447414007137651
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
Request to load LoRAs [('sd_xl_offset_example-lora_1.0.safetensors', 0.1), ('GeorgiaDogUN_R.safetensors', 0.98), ('None', 0.1), ('None', 0.1), ('None', 0.1)] for model [Z:\Fooocus\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors].
Loaded LoRA [Z:\Fooocus\models\loras\sd_xl_offset_example-lora_1.0.safetensors] for model [Z:\Fooocus\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors] with 788 keys at weight 0.1.
Loaded LoRA [Z:\Fooocus\models\loras\GeorgiaDogUN_R.safetensors] for model [Z:\Fooocus\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors] with 722 keys at weight 0.98.
Requested to load SDXLClipModel
Loading 1 new model
unload clone 1
[Fooocus Model Management] Moving model(s) has taken 1.18 seconds
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] GeorgiaDog is laying in the grass by the pond, bright, inspired, highly detailed, intricate, sharp focus, cinematic light, beautiful, classic, designed, vivid colors, open background, symmetry, contemporary, fresh, fantastic, creative, positive, unique, pure, attractive, elegant, perfect, cute, very coherent, artistic, color, illuminated, outstanding, epic
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] GeorgiaDog is laying in the grass by the pond, bright, inspired color, intricate detail, cinematic, clear, crisp, sharp, rich, gorgeous, stunning, highly detailed, romantic, designed, vivid, beautiful, infinite, symmetry, light, atmosphere, perfect, ambient, magic, pretty, scenic, thought, professional, best, artistic, winning, new, novel
[Fooocus] Encoding positive #1 ...
[Fooocus Model Management] Moving model(s) has taken 0.11 seconds
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
Preparation time: 7.48 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.02916753850877285, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 2.81 seconds
100%|███████████████████████████████████████████████████████████████████████████████| 30/30 [00:10<00:00, 2.92it/s]
Image generated with private log at: Z:\Fooocus\outputs\2023-11-13\log.html
Generating and saving time: 14.13 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.02916753850877285, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 1.56 seconds
100%|███████████████████████████████████████████████████████████████████████████████| 30/30 [00:10<00:00, 2.90it/s]
Image generated with private log at: Z:\Fooocus\outputs\2023-11-13\log.html
Generating and saving time: 12.90 seconds
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
[Fooocus Model Management] Moving model(s) has taken 0.39 seconds
Total time: 37.85 seconds

@lllyasviel
Copy link
Owner

thanks for the file. looking at it

@lllyasviel
Copy link
Owner

try 2.1.806

@lllyasviel lllyasviel reopened this Nov 14, 2023
@scsp85
Copy link
Author

scsp85 commented Nov 14, 2023

It’s working again! Thank you for the quick fix.

@scsp85 scsp85 closed this as completed Nov 14, 2023
@Dsnaps77
Copy link

Dsnaps77 commented Dec 8, 2023

Hello! Im able to get checkpoints working but have had zero luck with LORAs. No errors pop up and no matter how much i turn on the weight nothing happens. Ive tried 9 different ones but still no luck. I have the latest versions of Fooocus installed. Did a re install, read multiple guides and cant get them to load :( any help is appreciated.

@HiProAI
Copy link

HiProAI commented Dec 27, 2023

try removing some of the default styles in fooocus which are overriding the lora, seems like if you remove the styles it seems to work. some of them anyway

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants