Alpha channel "inline" on the image

[bradh]

It looks like the alpha channel is generally handled as an auxiliary image. In the external API, it appears that heif_image_handle_has_alpha_channel() can return false even if there is an available alpha channel. I think this is the case I'm hitting for the uncompressed codec, where the alpha component can be mixed in with the other values (pixel, component, row, etc).

Should HeifContext::has_alpha check for an inline channel (e.g. call HeifPixelImage::has_channel(heif_channel_Alpha) - although maybe it should also check for interleave with a heif_chroma value that includes alpha)? Or should the check be deeper (e.g. have get_alpha_channel() also check for an inline channel, and construct the required Image)?

I see there was earlier work at #708. This seems related but distinct, allowing for the comments at

libheif/libheif/heif.cc

Lines 827 to 841 in e4d554a

	int heif_image_handle_has_alpha_channel(const struct heif_image_handle* handle)
	{
	// TODO: for now, also scan the grid tiles for alpha information (issue #708), but depending about
	// how the discussion about this structure goes forward, we might remove this again.
	return handle->context->has_alpha(handle->image->get_id()); // handle case in issue #708
	//return handle->image->get_alpha_channel() != nullptr; // old alpha check that fails on alpha in grid tiles
	}
	int heif_image_handle_is_premultiplied_alpha(const struct heif_image_handle* handle)
	{
	// TODO: what about images that have the alpha in the grid tiles (issue #708) ?
	return handle->image->is_premultiplied_alpha();
	}

[farindk]

We probably have to extend HeifContext::has_alpha() with a special case for uncompressed images, like there is a special case for grid images. Using HeifPixelImage::has_alpha() probably won't work because we cannot use that before the image has been decoded. But we can get the information from the cmpd box, if I see that correctly.

[bradh]

We probably have to extend HeifContext::has_alpha() with a special case for uncompressed images, like there is a special case for grid images. Using HeifPixelImage::has_alpha() probably won't work because we cannot use that before the image has been decoded. But we can get the information from the cmpd box, if I see that correctly.

Would it make sense to extend the plugin API for this? I think JPEG 2000 can do an alpha channel inline, and if we want to wrap something like JPEG-XL in the future, that could also be a consideration.

While we can get information from the cmpd box, it is possible that there are multiple images, and none of them actually reference that entry - that is explicitly allowed in 23001-17 Section 5.1.1. Working through the uncC box, and then potentially special cases in cpal and cpat seems like a bit more than HeifContext should be doing. So maybe delegating off would be cleaner?

[farindk]

Working through the uncC box, and then potentially special cases in cpal and cpat seems like a bit more than HeifContext should be doing.

Yes. My plan is to refactor HeifContext into a collection of "parser" classes that handle reading/writing different aspects, e.g. images, sequences, ... Then, since the uncompressed codec is a bit special, it could have a parser class of its own, derived from the image parser. HeifContext is then just a collection of the active "parsers" for the current file.

But for the moment, we can just add this to HeifContext::has_alpha(). I can refactor that later.

Would it make sense to extend the plugin API for this? I think JPEG 2000 can do an alpha channel inline

Maybe. I also have added h265 decoding with embedded alpha stream some time ago to libde265. I've never seen this used for HEIC images, but there are some video streams coded like this and it might also be valid for HEIF.