A node based GUI?

[SMUsamaShah]

I have this idea for quite a while (since I know about this project) and have looked into it a bit too but never got the courage to actually do it, not very good at it anyway. Just putting it out here to see if it is really that useful.

What if this tool was a node based interface or had that as an alternative UI? Would it have made it more convenient for everyone. Only thing I am sure about is that all components would have become tiny modules which you could connect and visually see your pipeline too.

For example you would have nodes to engineer prompts. Then add further prompts on a result. Or expand (outpaint) a result or apply img2img on it. All of these steps could have been visible as previews.

I thought there has to be something like this already. I did find some tools like that. There are desktop applications but some web applications too. https://gist.github.com/SMUsamaShah/c923a0af4543ee2746979328825da236
2, 5 and 7 from this list are interesting ones.

I really think there should exist a node based GUI for SD. If there is something like that already I don't about it.

[seijihariki]

Hi, do you mean node-based blender style? If so, could you give some basic examples of nodes that would exist, and their inputs/outputs?

[zero01101]

so ignoring the heresy of mentioning a library in this staunchly, aggressively vanilla javascript only project, https://flume.dev/ is a neat library. that being said, i STILL really don't think there's enough "composable complexity" in what openOutpaint does that a node editor would provide any benefit, especially if each node has to wait for the previous in the chain to finish (or at least partially finish) processing before anything contextually relevant can be done in the next hop down.

like check out flume's examples of other node interfaces: they're all things that can be nearly instantly computed, or are a giant pain to manually compose
or something like reaktor

i'm probably just a grumpy old man but it just really feels like it's more cool than useful to how openOutpaint works, especially coming from a (somewhat modular) synth background and learning a proper respect for feedback :| it just feels like it would be better served as its own project? idunno, i'm not trying to say "absolutely not" or anything, i just really don't see the benefit personally but again, probably just grumpy old man ;)

[seijihariki]

I agree that it would probably do better as a separate extension. It simply doesn't seem to fit the current openOutpaint flow.

[SMUsamaShah]

I also agree that it can be a separate tool and cannot think of a way to integrate into openOutpaint.
https://nimp.app/ seems like a really nice candidate for that.

[seijihariki]

Hey, @SMUsamaShah, don't know if you keep watch of the stable diffusion subreddit, but this:
https://www.reddit.com/gallery/10lzgze
Seems to be nearly exactly what you were wanting, and it's eons ahead of whatever whole could probably pull off using the automatic1111 webui's API.

It seems the UI is this one:
https://github.com/comfyanonymous/ComfyUI

It could be a bit of a pain to switch UIs every so often to use this, but it may be an okay workflow until something comparable is useable in automatic's. Maybe there is an extension somewhere that does this.

[zero01101]

very highly seconding comfyUI holy crap that set conditioning area process is BRILLIANT and yeah, this is precisely where a node editor is useful lol; we'd NEVER be able to hook that deeply into the actual diffusion process in this app ;)