Different output resolutions: 512, 256, 128

[flamed0g]

Feature description

Hi there,

Currently the inswapper_128 model also restricts the output quality to 128 which often results into blurry/poor quality. However, @Hillobar from Rope managed to get a higher output resolution (up to 512) using the default inswapper_128 model.

I am not too sure yet how but it looks like the magic happens in this commit.

It would be awesome to also have higher output resolutions to improve the quality drastically.

[StellarBeing25]

I was going to open a feature request for the same. Please implement this technique to increase swap resolution and remove the face restore options from the main node since a separate face restore node is already included in the reactor.

[not-ski]

Here's the relevant code (I think) btw: https://github.com/Hillobar/Rope/blob/main/rope/VideoManager.py#L682

[RedDeltas]

I second this - I was about to create a new issue when I saw this one

[Mozoloa]

I've been digging through the code, and looks like rope is hijacking the whole inswapper process at the tensor scale. In reactor we just pass the original image with some coordinates for the face + embeddings from the facemodel and just send it to inswapper.

The guys at Facefusion seem to have rebuilt the inswapper model for each resolution boost calling them pixel boost, but they're paying for now.

Implementing here seems tough for my weak knowledge in all of this

[Gourieff]

Done in 0.5.1 alpha1 in some way https://github.com/Gourieff/comfyui-reactor-node#051-alpha1 feel free to test guys!
Big thanks @not-ski for the implementation of the idea via PR #321!
It's not exactly the same as here

Here's the relevant code (I think) btw: https://github.com/Hillobar/Rope/blob/main/rope/VideoManager.py#L682

but it does the job very well

[not-ski]

Thanks @Gourieff for merging my work into Reactor :)

To be clear, my approach is (I believe) quite different from what Hillobar is doing in Rope. I am simply applying the restoration model to the 128px swapped face before pasting it back into the image, whereas Rope appears to utilize some pixel-shifting magic to upscale the 128px face to 512px (as opposed to using a restoration model).

One of these days, I'll take a crack at implementing the latter into Reactor; I think combining this with the new restoration approach from #321 could produce some amazing results!

[Mozoloa]

Yep, Rope is doing some subsampling/shifting and reassembling that works quite well, I don't think that's it, but yeah, them combined could be very good

[not-ski]

Alright so after looking more closely at Rope's implementation, and trying to implement it in Reactor, it's really quite tricky (and clever).

Basically what he's doing is taking the original face, resizing it to 512px, and then splicing by every fourth pixel. This essentially "reconstructs" the face at 128px. He runs the swapper on this reconstructed/spliced face.

This is done 16 times in total (pixel shifting by 1 every time, in both dimensions) and the swapped face is then spliced back into a 512x512 canvas, matching the "spots" where the pixels were spliced from the original face.

I'm not sure if this explanation makes sense honestly, but if you look at the code that I referenced in my earlier comment I think you'll see what I mean. As someone else said, this involves some very significant hijacking of the inswapper process, and I haven't quite figured out how to replicate it in Reactor yet.

[not-ski]

@Hillobar is there any chance you could help with this a little bit? I would love to implement the pixel-shifting approach to Reactor :)

[MYSTERlOUSGUY]

Perhaps this can help.

[StellarBeing25]

@Gourieff @not-ski Any update on Rope's pixel-shift upscaling implementation in Reactor?

[not-ski]

@Gourieff @not-ski Any update on Rope's pixel-shift upscaling implementation in Reactor?

@StellarBeing25 I did spend some time trying to implement this in Reactor awhile back, but I think it's a bit beyond my pay grade lol, sorry!