[Issue]: Can not reproduce images after restarting the UI

[oshimoto]

Issue Description

Hi! I have noticed a couple of days ago that when I restart the UI (server) I can't reproduce the images I generated before restarting.
To reproduce the issue :

1. generate an image using a conforming sampler
2. generate the same seed again - now you have the exact same image
3. restart the UI
4. generate with the same settings again - and now you will get a slightly different image

When I first noticed this issue I was on commit eae813b. I upgraded to 9740b9d and tested again, the issue still exists.

Version Platform Description

Win 10,
version 9740b9d,
3070ti

Acknowledgements

• I have read the above and searched for existing issues

[vladmandic]

if unsure, ask question in discussions first. this is not an issue. and you've provided pretty much no reproduction info as such.
and yes, there are tons of things that can contribute to results variations.

[Davikar]

Check if you have token merging on.
I think there was a settings change recently that removed the on/off toggle, so mine was turned on by default.

[oshimoto]

Yes token merging is turned off.

[Learn2o]

Thank you for mentioning about Token Merging. I was scratching my head trying to figure out why I was suddenly having issues reproducing images that I had just generated moment before using the same parameters and seed. After disabling token merging I was able to reproduce images without issue.

[oshimoto]

Perhaps I didn't explain the issue clearly. I have used this UI for a couple of months now, and (almost) always when I use it I have been able to reproduce images just fine. After updating around 9-16 June this ceased to be the case. Now every time I restart the server the images are different. I have double checked all settings. I will upload screenshots of the settings later today when I get home.

I can still reproduce an image while I have not closed the UI. For example I generate an image using whatever propmt with a sampler I know is conformative. Next I generate an image with the exact same prompt, settings etc..
Now I have an EXACT replica of the previous image = I have successfuly reproduced the previous image
Then I shut down the server and start it again (without updating and without even closing the webpage).
Now I generate again with the exact same settings as above(they have not changed, because the webpage was not closed).
The result is a SLIGHTLY DIFFERENT image = it was not reproduced.

This is happening every single time I restart the server, not just once. I asume this became an issue after I updated between 09 -16 June. Before that is it was working fine.

[vladmandic]

how slightly?
which cross-optimization method are you using?

After updating around 9-16

this can be due to code change, but quite possibly because a lot of system default settings have changed - and there is tons that goes into reproducibility.

[oshimoto]

Yes but I can not reproduce an image created using commit 9740b9d again using the same commit 9740b9d. That is the whole problem! I will write in detail later tonight.

[vladmandic]

I understand what you're saying, but you've provided no actionable info.

[oshimoto]

I was just to about to start uploading settings screenshots and comparison images when I noticed the issue was gone :) Even after restarting the server images are reproduced perfectly.

As far as I can tell it was fixed when yesterday evening I deleted all my packages in python folder(long story :D, unrelated)
So today the UI downloaded some of the packages it was missing. I assume that is what fixed it, while previously it has been bugging me for 5 days.

Sorry for wasting your time on this one. Thanks for your help anyway!

[afbagwell]

I just ran into this issue myself and saw the discussion. I'm using the latest c90e996

I've tried several instances of re-doing images I've done before, importing them in through image processing and then sending to the "From Text" tab. Double-checked everything including the hash of the model being used. (It's one of my own.)

Sometimes the differences are subtle. Like a lamp will have a slightly different bulb design or a background character will be looking a different direction. Sometimes the changes are strikingly different. Same character, just different stance, background, etc. Sometimes there appears to be no difference at all.

Here's two examples. In this case both were generated on separate runs one right after the other. No parameters were changed. Variations are not turned on. No dynamic prompts are being used.

Here's the meta-data

Steps: 30 | Sampler: DPM++ 2M Karras | CFG scale: 9 | Seed: 393979067 | Face restoration: CodeFormer | Size: 512x680 | Model hash: 28adcae91f | Model: DreamNovaLifeLike | Denoising strength: 0.25 | Clip skip: 1 | Version: c90e996 | Token merging ratio: 0.5 | Token merging ratio hr: 0.5 | Parser: Full parser | Hires upscale: 2 | Hires upscaler: 4x_foolhardy_Remacri | Lora hashes: "epiNoiseoffset_v2: d1131f7207d6" | Score: 7.55 | Score: 7.35

If you want I can email you the images with embedded metadata as I'd rather keep my prompts private.

[vladmandic]

that might actually be the case as how lora works is to overwrite model layer on load and then restore model layer on unload - but if unload logic is not working, you'd get some lora effects in subsequent images even if lora is not specified in prompt.

i'm no longer maintaining original lora code and new planning a full switch to different interpreter for loras in the near future.
can you try using it? settings -> extra networks, set both options and restart server:

• Use LyCoris handler for all Lora types
• Disable built-in Lora handler

[afbagwell]

Tried your suggestion

Both of these have identical prompts using <lora:epiNoiseoffset_v2:1.0>
Metadata: Steps: 30 | Sampler: DPM++ 2M Karras | CFG scale: 9 | Seed: 89898902 | Size: 512x680 | Model hash: 28adcae91f | Model: DreamNovaLifeLike | VAE: vae-ft-mse-840000-ema-pruned | Clip skip: 1 | Version: cccae90 | Parser: Full parser | Score: 8.08
Used embeddings: bad-hands-5 [10ca]

[afbagwell]

Honestly at this point I think it would be best for me to do a re-install before testing this again. It's probably about time I did that to eliminate any possibility of some garbage left behind from installing and uninstalling various extensions between updates to the main repo.

[afbagwell]

OK I did a clean install of the latest SD.Next without adding any extensions and ran tests both with and without the Lora used in the images above.

Regenerating the same image from the same prompt produces different output, even for a simple prompt like:

Prompt: abstract painting, colorful, highly detailed, fine lines
Negative prompt: childish, poorly drawn

If you have an image with a significant amount of detail the effect is much more pronounced. And if you are using a Lora the effect appears to be a lot worse.

I know this wasn't always the case. Sometimes in testing image output I'd generate the same image at various points. When I use an image viewer program and advance through the list, these identical images appear remain static when loading the next file. When you do the same with the images I've been making in these tests, you can see the slight changes as each image updates from the previous.

WRT to Lora vs LyCoris. With the Lora extension enabled, yes, there's a problem that appears to be the Lora getting applied to subsequent images in the same generation set when it isn't part of the prompt. Switching over to LyCoris and disabling the Lora extension appears to fix this problem.

[vladmandic]

glad that at least lyco works better than lora in this regard. the rest, i believe you, but i don't know what to do about it.

[Z1-Gamer]

Hi. I also wanted to talk about this problem. I have used SDNext since February. The first time i noticed this problem was at the end of May. Since then i have tried many things to fix it but at the end for the last month i have used ComfyUI because of this issue. The exact issue is that every time i restart SDNext i cannot generate an identical image compared to prior to the restart. I will detail more below.
However Yesterday i decided to clean up everything and try again so here is what i did:

1. Deleted SDNext
2. Uninstalled and Deleted Python and all cache, pip etc. all related folders.
3. Installed Python
4. Installed SDNext (commit was 31e3bb0 i think)

So now i have a fresh install, i did not touch any of the settings except to set SDP-NO-MEM for reproduction.
I put a simple prompt and generated an image with Euler.
I generated another image (without changing any settings or prompt) to check if they are the same, They were identical.
I closed SDNext (CTRL+C on the command prompt window) and started it again
I refreshed the web page and clicked on read last generational settings
I generated again, and this time it's not identical while it should be because it's using all the same setting and prompt as the last 2 images. The only difference being that SDNext has been restarted (BTW if you restart it by using the "restart" option in the web page this issue is not present. Only if you completely close the command prompt window and start it again. Sadly this doesn't help because you can't keep it running forever and ever, and whenever you close it, the next time you start it you will not get the same image.

I am sorry i can't provide screenshots because i did not have disk space left and already uninstalled this instance so i can test with Automatic1111 1.5.1. With it i did 2 tests this morning and this problem is not present (for now).

I don't think this is a problem on my end because both ComfyUI which i have used for the past month and Auto1111(need more testing for Auto1111 but so far..) don't seem to have this issue, and because i did a complete fresh install of SDNext for my final test yesterday (python included).
I have no idea what can be done, it's sad really because i love SDNext and it's my first choice if it weren't for this issue. Comfy is good too but my problem with it is that i often generate on my tablet (local network) and comfy is very hard to use on mobile.
I will try to free up some space later today or tomorrow so if you have an idea of something to try and fix this i will be happy to test or provide more info/examples. Just tell me what you need and i will test when i can. Cheers.