Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AUX images support #117

Open
bigcat88 opened this issue Jul 20, 2023 · 8 comments
Open

AUX images support #117

bigcat88 opened this issue Jul 20, 2023 · 8 comments
Labels
enhancement New feature or request

Comments

@bigcat88
Copy link
Owner

Divided #83 topic into two separate.

Briefly: I can easy add AUX image support whenI will have any non iPhone image, with different auxiliary_type (e.g. non hdrgainmap)

Or when I get information from here: strukturag/libheif/issues/926

Currently we can not determine image mode, and I need to know, can all aux images be decoded as grayscales ones or not.
Cause if not, then algorithm will be a little bit weird, need first to try to decode as Greyscale and if decodes fails, then decode it as RGB/RGBA...

@bigcat88 bigcat88 added the enhancement New feature or request label Aug 9, 2023
@johncf
Copy link
Contributor

johncf commented Oct 7, 2024

Hello. I'm trying to get the HDR gain map aux image, and I see aux image support is yet to be implemented...

with different auxiliary_type (e.g. non hdrgainmap)

Why non-hdrgainmap? Is HDR gain map not planned to be supported, or is it somehow already supported? (I couldn't find any details about this in this repo.)

@bigcat88
Copy link
Owner Author

bigcat88 commented Oct 7, 2024

No, at the moment AUX images are not supported (I don't want to write a conversion from 16/12/10 bit images to 8 bit for them) - I'm waiting for Pillow to support HDR images, they seem to be working on it.

As soon as Pillow starts supporting RGB(a) 10/12/16 bit, pillow_heif will immediately be refactored and support for AUX and other things will appear.

@johncf
Copy link
Contributor

johncf commented Oct 7, 2024

I'm waiting for Pillow to support HDR images, they seem to be working on it.

Are you saying that once Pillow supports HDR images, pillow_heif will be refactored to read HDR gain map and combine the SDR and gain maps into a single 10/12/16 bit HDR representation?

If that's what you meant, I wasn't referring to that. I simply wanted to read the HDR gain map as is, independently from the SDR image. From what I can see, libheif already provides a way to extract all AUX images. Would it be within pillow_heif's scope/interest to expose that feature?

@bigcat88
Copy link
Owner Author

bigcat88 commented Oct 7, 2024

Would it be within pillow_heif's scope/interest to expose that feature?

definitely yes, but I myself don't want to add reading of only part of AUX images (for example, add reading of gain map which are 8-bit and their reading is relatively easy to add)

I myself will only have time for the final refactoring, when Pillow will support this:

python-pillow/Pillow#1888


if you want and you have time, I will gladly accept and test the PR for this, even for supporting only part of the AUX image variants.

@johncf
Copy link
Contributor

johncf commented Oct 9, 2024

Great! I do have some time and started digging in https://github.com/johncf/pillow_heif/tree/aux-images

For now, I made a change that exposes all aux images through aux_image_list property in C (and .info["aux_images"] in Python) so that I can look at them properly. Here, I'm making an assumption that all AUX images are greyscale, which is likely wrong.

  1. Do you think it's good (or possible) to make a general implementation that exposes all (non-alpha, non-depth) aux images like this at the moment? (If heif_image_handle_get_preferred_decoding_colorspace works correctly on an aux image handle, then the above false-assumption could be fixed.)
  2. If it's not a good idea to expose all aux images as of now (due to various limitations), do you have a preference on how HDR gain map should be exposed? What I'm interested in is the gain map captured by an iPhone, which has aux_type as urn:com:apple:photo:2020:aux:hdrgainmap. Would this be an acceptable path for this: .info["apple2020_hdrgainmap"]?
  3. Do you know whether a single HEIF file can have more than 1 aux images having the same aux_type? (this can be accessed through himage.info["aux_images"][0].info["aux_image_type"]) If they're always unique, then .info["aux_images"] can be a dictionary from aux_type to image.

@bigcat88
Copy link
Owner Author

bigcat88 commented Oct 9, 2024

  1. You can make a list in info["aux"] - all "AUX" images (not all need to be supported at once, for example, to start we can support only 8-bit)
  2. Yes, why not. Alternative can be .info["hdrgainmap"]["apple2020"] or .info["aux"]["hdrgainmap"] and set "apple2020" as some property for easy filtering for example.
  3. AFAIK, yes, they can be. There is also "subtype" from libheif sources. If anything, you can always ask in the libheif repository, they know a lot there.

@johncf
Copy link
Contributor

johncf commented Oct 10, 2024

I made a PR (#297) with a slightly different approach that doesn't give any special treatment to Apple's way of doing things.

Out of topic: If you like that approach of doing things (i.e., expose only a list of IDs from the HeifImage, and provide a method to get the aux image, given an ID), would it be a good idea to do the same with thumbnails or depth images? (If so, please create an issue and tag me.)

@bigcat88
Copy link
Owner Author

Thank you for making PRs, I will review them tomorrow or a day after and ping you of course

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants