fix extra batch mode P Transparency

[w-e-w]

Description

fix

• [Bug]: Upscale Batch Proccess TypeError #6534

when enable_pnginfo using extra "batch" the existing_pnginfo will also be restored

note: issue is only in Batch process not single
I believe gradio image component already does a conversion once as a result this issue does not occur when using Single image

when converting a none RGB mode image to RGB some existing_pnginfo keys will also have to be converted to make it work
this is the case with a P mode image with Transparency key

the issue is we currently extract the existing_pnginfo "before" we convert the image into RGB mode

as suche the existing_pnginfo we restore later will have the old unconverted existing_pnginfo from P mode

in the case of P -> to RGB if Transparency : 0 key exist, it will have to be converted in to a 3 value tuple (0, 0, 0)
this is why some users are seeing error

*** Arguments: ('task(io7z85oegmsjbmy)', 1.0, <PIL.Image.Image image mode=RGBA size=119x152 at 0x1EAA924E8F0>, [<tempfile._TemporaryFileWrapper object at 0x000001EAA924E350>], '', '', True, True, 0.0, 2, 0.0, 512, 512, True, 'R-ESRGAN 4x+ Anime6B', 'None', 0, False, 1, False, 1, 0, False, 0.5, 0.2, False, 0.9
, 0.15, 0.5, False, False, 384, 768, 4096, 409600, 'Maximize area', 0.1, False, ['Horizontal'], False, ['Deepbooru']) {}
    Traceback (most recent call last):
      File "B:\GitHub\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "B:\GitHub\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "B:\GitHub\stable-diffusion-webui\modules\postprocessing.py", line 131, in run_postprocessing_webui
        return run_postprocessing(*args, **kwargs)
      File "B:\GitHub\stable-diffusion-webui\modules\postprocessing.py", line 96, in run_postprocessing
        fullfn, _ = images.save_image(pp.image, path=outpath, basename=basename, extension=opts.samples_format, info=infotext, short_filename=True, no_prompt=True, grid=False, pnginfo_section_name="extras", existing_info=existing_pnginfo, forced_filename=forced_filename, suffix=suffix)
      File "B:\GitHub\stable-diffusion-webui\modules\images.py", line 728, in save_image
        _atomically_save_image(image, fullfn_without_extension, extension)
      File "B:\GitHub\stable-diffusion-webui\modules\images.py", line 712, in _atomically_save_image
        save_image_with_geninfo(image_to_save, info, temp_file_path, extension, existing_pnginfo=params.pnginfo, pnginfo_section_name=pnginfo_section_name)
      File "B:\GitHub\stable-diffusion-webui\modules\images.py", line 584, in save_image_with_geninfo
        image.save(filename, format=image_format, quality=opts.jpeg_quality, pnginfo=pnginfo_data)
      File "B:\GitHub\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 2432, in save
        save_handler(self, fp, filename)
      File "B:\GitHub\stable-diffusion-webui\venv\lib\site-packages\PIL\PngImagePlugin.py", line 1364, in _save
        red, green, blue = transparency   #  <---- the unconverted transparency key from P mode is just 0, the not the expected (0, 0, 0)
    TypeError: cannot unpack non-iterable int object

---

the easiest solution I found is to convert the image to the to our desired mode before parseing the info with read_info_from_image()

I have tested with the 4 formats of images generated by webui (png jpg webp avif) the info (the parts we wish to preserve) read after conversion seems to be intact

---

test images
test images.zip
test from #15421 #6534
2 p mode image with and without transparency key

before the PR the image with transparency key should faile to save using extra batch
should be fixed after the PR

---

note
I'm not an expert on image formats it's it is possible this modification can potentially messed up other things

Checklist:

• I have read contributing wiki page
• I have performed a self-review of my own code
• My code follows the style guidelines
• My code passes tests

[Rabcor]

Worked for my sample files in #6534 (comment)

Not seeing any detrimental effects at all. It does change some behavior between batch and single image processing though.

Namely that batches now try to preserve alpha while single image processing does not, this is in my opinion an improvement though and maybe it'd be worth it to figure out a way to get transparency working on single image processing as well.

Original:

Single Image Upscale:

Batch Upscale:

[w-e-w]

about the single image
I know the cause and I have a way to fix it, but this method may brakes some extensions
if you're interested you can see https://github.com/AUTOMATIC1111/stable-diffusion-webui/tree/more-extra-transparency-fix-again

basically the cause of the losing of transparency for P mode images is due the the gradio convertion the input image into RGBA
so when it gets to our script is already a RGBA image, the
when a P image gets converted into RGBA the transparency key in removed as it is encoded into Alpha channel
up to this point is fine

next depending on the process module you use such as upscale
upscale is for RGB only so the output of upscale is RGB not RGBA
what has happened the transparency information is lost

it workd for Batch because we are the ones doing the converting from P to RGB, and since we saved the tansparency key befor converting and restore it after it's finished
transparency information is preserved

in this branch https://github.com/AUTOMATIC1111/stable-diffusion-webui/tree/more-extra-transparency-fix-again what I did is disable Gradios auto converting of image to RGBA, by disalbeing preprocess, this way it retruns the raw image in base64 string format, doing so loud us more control on the converting process and allowing the preservation of the transparency key

the issue is as far as I'm aware disalbeing preprocess applies for every component for that submit,
if there is currently any other extensions or modules that requires preprocessing (like if that moduls has a second image input)
they will also receive base64 as opposed to the image they're expecting, stuff wll break

so if there's a way of preserving the transparency key after it's after it has been converted to RGBA then things would work

---

another method is to set

stable-diffusion-webui/modules/ui_postprocessing.py

Line 15 in 1c0a0c4

extras_image = gr.Image(label="Source", source="upload", interactive=True, type="pil", elem_id="extras_image", image_mode="RGBA")

from RGBA bask to RGB
the change to RGB was done in

• Allow PNG-RGBA for Extras Tab #15334

---

yeah it's kind of ironic making RGBA possible for some modules actually breaks transparency for other image mode

[Rabcor]

yeah it's kind of ironic making RGBA possible for some modules actually breaks transparency for other image mode

Actually, here's an extra sample file to test this.

And here's the old sample file I am still using here as well:

Unmodified 1.9.3 baseline

With this, we have established that this commit: #15334

Is only enabling transparency for the reactor extension, not upscaling.

Upscale


Reactor

#15664 applied to 1.9.3

Identical to baseline.

Upscale


Reactor

#15334 undone for 1.9.3

Interestingly undoing the commit made it preserve transparency on one sample file but not the other when upscaling.

Upscale

Reactor

#15664 applied and #15334 undone for 1.9.3

Identical to above.

Upscale

Reactor

master...more-extra-transparency-fix-again changes applied to 1.9.3

One of the sample files still isn't transparent. Reactor successfully preserves transparency but fails to apply any changes to the face.

Upscale

Reactor

[w-e-w]

• With this, we have established that this commit: Allow PNG-RGBA for Extras Tab #15334
  Is only enabling transparency for the reactor extension, not upscaling.

no we have not established that

what that PR dose it's enabling possibility of "Proper" RGBA support for single image input for ANY module as long as the module "itself" support transparency

upscaller depending on the upscaler method you use, and most dose not support transparency (Alpha Channel)

the reason you were able to get desirable transparency results is mostly a combination of luck

your image is a P mode image
https://pillow.readthedocs.io/en/stable/handbook/concepts.html#modes

P mode: 8-bit pixels, mapped to any other mode using a color palette

which uses a different method of encoding transparency then RGBA
as far as I'm weird image is essentially a grayscale image with a color palette information encoded alongside
including the transparency of certain color palettes

the lucky part is if the color palettes transparency is save and re-apply back to the image you would get the transparency back
as long as the processing module you're using does not shift the color

you can test it with different upscaling methods the more a particular message shifts the color the more likely that the transparency will be ruined

some of the test image you sent, image is basically composed of non don't transparent color parts
and transparent background

img1	img2	img3

test of img1 with different upscaling methods

Lanczos	R-ESRGAN 4x+ Anime6B	R-ESRGAN 4x+	16x-ESRGAN
transparency preserved with artifacts	image is complete gone leaving only transparent background	because of color shift you got blocks of black	transparency is completely gone

if you are "lucky" and the color of the output image has the same color pallette as the input, it we then restore the info back transparency gets restored
if you're unlucky then you might have results ranging from getting artifacts to completely ruined

the issue is not with that PR it's issue with lots of modules does not support transparency

• the issue is not with Allow PNG-RGBA for Extras Tab #15334 adding the possibility of proper transparency support

in fact after taht PR it has better transparency support
as long as all the modules you use in the processing pipeline support transparency

---

the reason that PR accidentally break transparency support for certain images you are using
is because

the issue with gradio preprocess auto converting image into RGBA,
during during conversion the the partial transparency metadata you have in your P mode image
just converted to the more widely accepted and fully featured Alpha Channel
this conversion is done before we have a chance to preserve the origin transparency metadata

and when later down the pipeling after going through upstairs that does not support transparency (Alpha) is lost
and so when saving the image the transparency is lost

---

so the way I found to "fix" that is to take converting image into our own hands and not handled gradio
but the orthodox method of doing so requires disableing preporcessing for the entire call
the issues if we do so it has a potential of breaking extensions

what I would consider doing would be to patch gradio so that we have more contorl
but I'm hesitant to do this now because I'm not sure when will we be upgrading to gradio4.x,
because if we do so stuff like this sort of pathces will probably break

but the thing is I'm not entirely sure if doing all this is worth it as transparency is at best support finicky
depending on your image contents and the upscaling method you're using

---

• the intent of this PR fix extra batch mode P Transparency #15664
  is not providing transparency support, it is to fix P mode image with transparency information throwing TypeError: cannot unpack non-iterable int object and fail to save the image

personally I think it might be a good idea to just remove the transparency altogether because it could potentially ruin the image
but I figure it might be a good idea to still preserve it on off chance that if it works for certain images

[Rabcor]

Yeah, including transparency is never going to be harmful, it's pretty hard to make an opaque image transparent, but the reverse is extremely easy, so it's never a bad thing.

But yeah, this PR solves the error it set out to solve perfectly.