Preserving the style in replaced face

[Hjelmdallermann]

Greetings!
I don't have much experience with SD and other things, so excuse my ignorance regarding some things. Overall this extension is extremely impressive, in large part because of how easy it is to operate.

I use Reactor to make generated images resemble some of the characters. I've been trying to create Harrier from Disco Elysium, and managed to find a decent combination of model, prompt and LORAs to replicate the vibe of Disco Elysium artstyle. For example,

Then, I use the Reactor to apply the face via img2img:

And the face looses all the style of the original image, looking very realistic instead (which might be coming from the SD checkpoint/base model?) Same thing happens in txt2img.

I initially thought that Reactor somehow uses the currently selected model and generates a face using the "reference" image. Seems that it works somewhat differently, since the inpainted face lacks the effects of LORAs/parts of the prompt.

Is there any way I can preserve the style in face-swapping?

Some of the stuff that I use:

Steps: 35, Sampler: DPM++ 2M Karras, CFG scale: 5, Seed: 2429885197, Size: 512x768, Model hash: d37263937f, Model: epicphotogasm_zUniversal, Denoising strength: 0.4, Hires upscale: 1.5, Hires steps: 15, Hires upscaler: 4x_NMKD-Superscale-SP_178000_G, Lora hashes: "Johannes Vermeer style: 43b93181a2e8, add_detail: 7c6bad76eb54, epi_noiseoffset2: d1131f7207d6, DiscoElysiumStyle: 864a597a0601, DE_Alexander_Rostov: e42e560bbb9d", Version: v1.6.0-2-g4afaaf8a

And also, Reactor:

Restore Visibility: 1
Codeformer Weight: 0.5
Upscaler: 4x_NMKD-superscale
Scale By: 1.5
Model: inswapper_128

Thanks in advance!

[Gourieff]

Hello!
It happens because of the face restorers influence in the final result
ReActor uses 128x128 inswapper model to swap the face, that's why we need to restore the face after we make a swap
Nothing can be done with this right now - only one inswapper model is available for users :(
So, CodeFormer tries to make a face as more realistic as it can, restoring a lot of details, try to play with the "CodeFormer Weight" parameter to change its affection to the face
Or try GFPGAN, it's more appropriate for art images
Also you can try to reduce the "Restore Face Visibility" parameter a little

[Hjelmdallermann]

Thanks for the detailed answer!
I tried to play around with weights of the CodeFormer, but it mostly seems to be a slider between quality of restored face and "original" generation; it helps, but for the "heavy" stylization in this case it might not be enough.
If you don't mind, I had a few more follow-up questions? It's not just about the style, I'd really like to get a better understanding of how Reactor works :D

1. I suppose using something different other than 128x128 inswapper would not give much of an effect, since it's still face restoration that neglects the style after? (I feel like I saw some comparisons of inswapper models, I thought there's more than one available...)
2. So generated/inpainted/restored face after face swap is not generated by the model used in original generation? I.e. that's 2 different models in the resulting image: the one that I use with bunch of LORAs, and CodeFormer to generate a face from the reference image?
3. I saw there other face-swapping options; I haven't been able to research them yet, but do they have the similar limitation? I.e. that the inpainted face is detached from the base model that does the generation? I really enjoy Reactor for it's ease of use, so I'm not sure I want to try setting up more complex solutions...
4. Is there a way to increase overall "weight" of inpainted face, so that it would resemble the reference more? Sometimes it's cool how subtle Reactor can make the resemblance, but other times it would be great to increase it even more.

Would really appreciate answers to any of those, thanks a lot!

[Moussack]

3. I've tried all faceswap model and IMO inswapper model still beats all of them.