Using -define tiff:alpha=associated results in modified semi-transparent areas

[coeing]

ImageMagick version

7.1.0-60

Operating system

Windows

Operating system, version and so on

Windows 11

Description

There is already a longer discussion about it: #2839

The short version:

• Not using -define tiff:alpha=associated results in the correct output, but unfortunately Adobe InDesign does not handle the background as transparent in that case (Photoshop and Gimp do)
• Using -define tiff:alpha=associated makes the background transparent, but modifies the semi-transparent areas of the image
• There is a flag in Gimp when exporting an image as a TIFF which seems to have the same function as the IM define and it gives the correct output without any modifications to semi-transparent pixels in both cases
  

Steps to Reproduce

• Run magick convert original.png -define tiff:alpha=associated associated.tiff
• Semi-transparent pixels are less white than in the original image

Images

[snibgo]

Using "-define tiff:alpha=associated" makes the background transparent, but modifies the semi-transparent areas of the image

Associated alpha "premultiplies", ie it multiples each colour sample by alpha before saving. If Associated does what you want apart from the premultiplying, then I suppose a workaround is to first divide the colour samples by alpha. Untested:

magick in.png -channel RGB -fx "u/a" +channel -define tiff:alpha=associated -define quantum:format=floating-point -depth 32 out.tiff

For non-opaque pixels u/a will be greater than 1.0, so your IM must be HDRI for this to work. (See magick -version.) I also save this as a floating-point TIFF.

When IM or Gimp reads this image, it is too bright. But it might be a workaround for Adobe InDesign.

If this workaround works, then it would seem to be a problem with InDesign.

[coeing]

@snibgo Thanks a lot for your quick reply!

Unfortunately this does not work. It makes the colors brighter, but does not fix the semi-transparent areas ("revert-premultiply" is your script):

I tried some things like this as a workaround, because my thinking was similar to yours, that the premultiplication does change the color values. But none of my tryouts worked either.

The problem with InDesign is, that it does not seem to like non-associated tiff files. In Photoshop and Gimp those are correctly displayed. So I cannot blame IM for this.
But using the -define tiff:alpha=associated command seems to have an issue because of the color/alpha changes of semi-transparent areas. As I wrote, the premultiplication done in Gimp results in the correct result, so it looks like a bug in the IM implementation.

[snibgo]

I've had a quick look at IM's source, and I don't think IM does any alpha associated or unassociated processing when saving TIFF files. It just sets a flag. I suppose the tiff library (libtiff) uses that flag.

EDIT: Another experiment for you to try: -alpha associate and -alpha disassociate. These change pixel values. So try:

magick in.png -alpha associate -define tiff:alpha=associated -define quantum:format=floating-point -depth 32 out1.tiff

magick in.png -alpha disassociate -define tiff:alpha=associated -define quantum:format=floating-point -depth 32 out2.tiff

[fmw42]

In IM 7, use magick, not magick convert.

[coeing]

I've had a quick look at IM's source, and I don't think IM does any alpha associated or unassociated processing when saving TIFF files. It just sets a flag. I suppose the tiff library (libtiff) uses that flag.

EDIT: Another experiment for you to try: -alpha associate and -alpha disassociate. These change pixel values. So try:

magick in.png -alpha associate -define tiff:alpha=associated -define quantum:format=floating-point -depth 32 out1.tiff

magick in.png -alpha disassociate -define tiff:alpha=associated -define quantum:format=floating-point -depth 32 out2.tiff

@snibgo Thanks for the additional tryouts, but they did not succeed unfortunately. -alpha associate results in the same image than without it, -alpha disassociate will give a pure white/opaque background.

By the way: The changes to the semi-transparent pixels are also visible in the Windows image preview and in Gimp as well. Probably in other image viewers as well, so it does not look like an InDesign issue.

Can you point me to the IM source code, so I can have a look there as well?

[coeing]

I put my detailed IM commands and the Gimp comparison into the discussion: #2839 (reply in thread)

Would be great if anybody could try to reproduce the issue on her/his machine, so we can be sure that it is not a local, e.g. Windows or version problem.

[snibgo]

Can you point me to the IM source code, so I can have a look there as well?

On this page, click on "Code" at top-left. Reading and writing tiffs is in coders/tiff.c.

[fmw42]

Using IM 7.1.1.37 on Mac OSX Ventura with libpng 1.6.39 and libtiff 4.6.0, the following produces virtually identical input PNG and output TIFFs

magick original.png -define tiff:alpha=unassociated -compress LZW alpha_unassoc_lzw.tiff
magick original.png -define tiff:alpha=undefined -compress LZW alpha_undef_lzw.tiff

magick compare -metric rmse original.png alpha_unassoc_lzw.tiff null:
6.77417e-13 (1.03367e-17)
magick compare -metric rmse original.png alpha_undef_lzw.tiff null:
6.77417e-13 (1.03367e-17)

What are your versions of libpng and libtiff?. (especially the latter)

[coeing]

Using IM 7.1.1.37 on Mac OSX Ventura with libpng 1.6.39 and libtiff 4.6.0, the following produces virtually identical input PNG and output TIFFs

magick original.png -define tiff:alpha=unassociated -compress LZW alpha_unassoc_lzw.tiff
magick original.png -define tiff:alpha=undefined -compress LZW alpha_undef_lzw.tiff

magick compare -metric rmse original.png alpha_unassoc_lzw.tiff null:
6.77417e-13 (1.03367e-17)
magick compare -metric rmse original.png alpha_undef_lzw.tiff null:
6.77417e-13 (1.03367e-17)

What are your versions of libpng and libtiff?. (especially the latter)

I have libtiff 4.6.0 and libpng 1.6.43:

      PNG* PNG       rw-   Portable Network Graphics (libpng 1.6.43)
             See http://www.libpng.org/ for details about the PNG format.
      TIFF* TIFF      rw+   Tagged Image File Format (LIBTIFF, Version 4.6.0)
             Compression options: None, Fax/Group3, Group4, JBIG, JPEG, LERC, LZW, LZMA, RLE, ZIP, ZSTD, WEBP

The unassociated and undefined TIFFs look like the original for me as well, but the associated TIFF does not. Which is what I am hoping to get.
The problem with the unassociated and undefined TIFFs is not that their colors are off, but that their background in Adobe InDesign is not transparent. If this is something that can be fixed I would happily use unassociated/undefined, but for me it looks like a bug that the colors of the semi-transparent pixels in the associated version are different than in the original image.

[fmw42]

The backgrounds look transparent to me. Perhaps the issue is with your Adobe inDesign and how it reads the Tiffs, not with Imagemagick. In my Adobe Photoshop, the backgrounds are transparent. So it seems to me that it is an inDesign issue. Check with Adobe!

[coeing]

The backgrounds look transparent to me. Perhaps the issue is with your Adobe inDesign and how it reads the Tiffs, not with Imagemagick. In my Adobe Photoshop, the backgrounds are transparent. So it seems to me that it is an inDesign issue. Check with Adobe!

@fmw42 Yes, that's probably the case, in Photoshop and Gimp the backgrounds are transparent for me as well. That's why I want to use the associated image which background is transparent in InDesign as well.

I just did the comparison between the original image and the associated image in the way you compared the original to the unassociated/undefined versions:

magick compare -metric rmse original.png .\im-compress-alpha-associated.tiff null:
3340.92 (0.0509791)

So it looks to me that the images are really different. Could you check if an associated version that you generated on your machine is different as well? If so, it really looks like a bug in ImageMagick or some library that ImageMagick is using, doesn't it?

[snibgo]

So it looks to me that the images are really different.

We expect them to be different, as one has unassociated alpha and the other has associated alpha.

[coeing]

So it looks to me that the images are really different.

We expect them to be different, as one has unassociated alpha and the other has associated alpha.

Okay, too bad, that is not how I understood the option. Is there some way to get associated alpha, but unchanged color values for the semi-transparent areas with a different command? Gimp can make it happen, too:

magick compare -metric rmse original.png .\gimp-not-save-color-values.tiff null:
33.7073 (0.000514341)

The result is not 100% accurate, but close enough.

This is the option that Gimp has for it, maybe it helps to understand what I would like to achieve:

[coeing]

Just found the pieces of code that Gimp does when the color values should not be saved:
https://github.com/MichaelMure/gimp-plugins/blob/b589aaede0e9703e4b4f7a106e7b6a29622836d1/common/file-tiff-save.c#L996
https://github.com/MichaelMure/gimp-plugins/blob/b589aaede0e9703e4b4f7a106e7b6a29622836d1/common/file-tiff-save.c#L818

I guess the second link just sets a flag in the tiff, like -define tiff:alpha=associated does? Than maybe doing something like in the first link with IM would create the same result as in Gimp?

Just throwing around some ideas 🙂

[fmw42]

From GIMP: This looks like IM using unassociated in place of associated for alpha.

      if (tsvals.save_transp_pixels)
        extra_samples [0] = EXTRASAMPLE_UNASSALPHA;
      else
        extra_samples [0] = EXTRASAMPLE_ASSOCALPHA;

This looks like premultiplying or not.

            case GIMP_RGBA_IMAGE:
              for (col = 0; col < cols*samplesperpixel; col+=samplesperpixel)
                {
                  if (tsvals.save_transp_pixels)
                    {
                      data[col+0] = t[col + 0];
                      data[col+1] = t[col + 1];
                      data[col+2] = t[col + 2];
                    }
                  else
                    {
                      /* pre-multiply rgb by alpha */
                      data[col+0] = t[col + 0] * t[col + 3] / 255;
                      data[col+1] = t[col + 1] * t[col + 3] / 255;
                      data[col+2] = t[col + 2] * t[col + 3] / 255;

A software engineers will have to comment further. But to see it looks like GIMP is doing the same as IM when setting alpha to unassociated and whatever else (not premultiply with alpha) that is needed when setting alpha to unassociated. So it looks to me that GIMP and IM are doing the same thing. But you could review the IM code to verify that.

Also I showed in a previous comment that Imagemagick is doing a pixel-by-pixel compare of tiff with png and the results are identical.

[coeing]

Also I showed in a previous comment that Imagemagick is doing a pixel-by-pixel compare of tiff with png and the results are identical.

Yes, unassociated images are identical to the original PNGs. But the associated images are neither identical to the original PNG nor the image that Gimp produces. The associated (!save_transp_pixels) one that Gimp produces is almost identical to the original PNG and is the version that I am trying to get out of IM (see #7578 (comment)).

[fmw42]

As snibgo said, the associated image has different alpha blending than unassociated and so would not be expected to look the same. You cannot have the transparency you want with associated. That is just not how it works. (GIMP is changing there associated to unassociated to display it the way you want)

[coeing]

As snibgo said, the associated image has different alpha blending than unassociated and so would not be expected to look the same. You cannot have the transparency you want with associated. That is just not how it works. (GIMP is changing there associated to unassociated to display it the way you want)

Are you saying GIMP is just changing the displayed image? This is something I cannot confirm. In #7578 (comment) I did the comparison with magick compare, so it has nothing to do with the way GIMP displays the image, but really is a about the way GIMP creates the associated image. Which is different than the way IM does it.

Of course I cannot decide which way is the correct one, but I'd like to achieve the version that GIMP generates with IM.

Do you have the possibility to generate a TIFF from the original image with GIMP? Maybe this helps to understand the differences between the generated associated images from IM vs. GIMP.

[coeing]

Yay, I finally arrived at my desired result! Not really sure why it works as it does, but of course I want to share it here to help other devs with the same problem or the IM devs to get a hint what is wrong with the default command.

So, this is the original command and result:

magick original.png -define tiff:alpha=associated -compress LZW im-compress-alpha-associated.tiff

magick compare -metric rmse .\original.png .\im-compress-alpha-associated.tiff null:
3340.92 (0.0509791)

After playing around a bit with the fx operation, I used the -channel operation. Even without an fx operation this gives the desired result:

magick original.png -channel R,G,B -define tiff:alpha=associated -compress LZW im-channel.tiff

magick compare -metric rmse .\original.png .\im-channel.tiff null:
33.7073 (0.000514341)

It is still not completely identical to the original, but it is identical to the image produced by GIMP:

magick compare -metric rmse .\gimp-not-save-color-values.tiff .\im-channel.tiff null:
0 (0)

So -channel R,G,B did the magic! Really unexpected, but I was so close to introduce GIMP into our pipeline, which is not necessary anymore. Thus I am really happy right now 🙂 Thanks @fmw42 and @snibgo for your comments, it helped thinking of other solutions 👍

[coeing]

For the sake of completeness, here is the original image and all generated ones:

generated.zip

[snibgo]

Thanks for solving this, and telling us about your solution, @coeing.

I don't know why "-channel R,G,B" solves the problem. Incidentally, the commas shouldn't be needed, this should be equivalent to "-channel RGB". Can you check that? Thanks again.

[coeing]

Thanks for solving this, and telling us about your solution, @coeing.

I don't know why "-channel R,G,B" solves the problem. Incidentally, the commas shouldn't be needed, this should be equivalent to "-channel RGB". Can you check that? Thanks again.

Yes, -channel RGB works as well. Happy to hear that I am not the only one who is surprised about the solution 😉