specs for axes coordinates information

[giovp]

hi,
w/ @LucaMarconato @kevinyamauchi we realized there is no way (or we didn't find it) currently to save axes coordinates information with the current specifications. Are there (or have there been) already some conversations about this and whether it could be included in the spec? This would be particularly useful for instance to save channel information (e.g. antibody name etc.) in multiplexed images.

Could this info potentially fit in the current spec for coordinate systems from @bogovicj (for instance, as additional metadata in the implicit Array coordinate systems).
Thanks!

[will-moore]

You can save channel labels (e.g. antibody name etc) & colours in the "omero" https://ngff.openmicroscopy.org/latest/#omero-md metadata.
This is supported by napari and vizarr viewers.

[jwindhager]

Hi! Working with multiplexed imaging data, I was running into the same issue. To me it feels wrong to store this type of information in the transitional omero metadata, because (a) we are not using Omero and (b) this type of information is used not just for visualization, but also during downstream analysis (cell segmentation, feature extraction, etc). Are there any plans to add axis coordinates directly to the multiscales metadata, specifically for "finite" (i.e., channel) axes (e.g. channel name)?

[will-moore]

There's an open PR at #138 to store axis coordinates, but I'm not sure how to use it to store channel names? cc @bogovicj?

In the meantime, I wouldn't let the "omero" name prevent you from storing channel names there. The name is more an indication that this representation matches the JSON format that OMERO uses in the web. But you don't have to be an OMERO user to use it.

[jwindhager]

Thanks @will-moore! Is the omero metadata specified somewhere? Or do we basically need to figure out the minimal required metadata by trial and error of individual tools at the moment (e.g. the napari plugin requires channels/*/color and channels/*/label, if I'm not mistaken)?

[will-moore]

See https://ngff.openmicroscopy.org/latest/#omero-md, although it's not as well documented as other parts of the spec.
The JSON schema for validating omero is at

ngff/0.4/schemas/image.schema

Line 53 in 642a7a1

"omero": {

which states that channel "window" and "color" are required.

The metadata read by ome-zarr-py and passed to napari-ome-zarr is at https://github.com/ome/ome-zarr-py/blob/db0f5dc6eea4ee1ca03598fee8542177e253be29/ome_zarr/reader.py#L357, which includes channel label, window, active and color.

[bogovicj]

Thanks @giovp, @LucaMarconato, @kevinyamauchi.

Summary: I think there is a way channel information could interact with coordinate transforms in a useful way, but most of it belongs in its own place.

This crossed my mind when expanding out the coordinate transform (ct) spec, and I intentionally ignored it at the time since that didn't seem to be the right place for it. I might be having a change of heart given how xarray works, but my preference now is to put the important imaging stuff (antibody, protein, wavelength etc) its their own place. Maybe consider naming every channel in a ct if we want to refer to them by name and not index.

There's two related ways I want to use cts, one of them feels "okay" for channel info, and the other feels "wrong."

indexing (wrong?)

ct's will let one index into the array data using the new coordinate system. For example, with physical coordinates:

give me the intensity at (10.5, 250.1, 12)nm, instead of at discrete index [55, 251, 33].

If axis information was used in a ct, I'd expect indexing with it to be possible, and that might be weird. This was what initially felt "wrong" to me.

But... it might not be. xarray has string-based indexing, so maybe it would be appropriate.

e.g. img.sel(c='mito-membrane') or similar

informational (okay?)

ct's also let you get information about a point in the pixel grid (if it is invertible).

What is the physical coordinate of the array index [55, 251, 33]? applies transformation (10.5, 250.1, 12)nm

This is the way that I see ct's making sense for, say, channel information.

What channel is at c=2? does lookup { "name": "mito-membrane", "reporter" : "TOMM20" } or whatever.

brainstorm

Here's my idea at what might make sense for a 3-channel image.

{
    "type" : "coordinate-information",
    "coordinates" : [
        { "name" : "ch0" },
        { "name" : "descriptive-name", "custom-field" : "optional", "Wavelength": 440 }
        { "name" : "mito-membrane", "reporter" : "TOMM20", "Wavelength" : 640 }
    ]
}

If this was used to make a coordinate system, I'd expect to refer to channels by their names, not their indexes.
It would be good to eventually give some direction about what information should go there (inheriting from
OME),
but that's "extra stuff" from the point of view of the ct.

Since the ct doesn't care about the other useful stuff (like wavelength), I don't like it and think it belongs elsewhere. If we want xarray indexing coordinates that map to strings could be cool, but I'd lean to putting all the other important stuff in a dedicated place.

[giovp]

thanks for the detailed reply,

but I'd lean to putting all the other important stuff in a dedicated place.

regarding this, I think there might be synergies with the way MITI does it,e.g. see the schema for channels

[jluethi]

Just saw this now. We're using the omero metadata to store channel names, some other channel identifier, rescaling values and colors and are very happy with it in Fractal :) I second the "don't let the omero name stop you" point from @will-moore . We identify our channels based on this metadata for processing and that works well.

If the channel names and other properties like wavelength identifiers etc. should go somewhere else, fine as well. I agree with @bogovicj though that they don't feel like transformation information

[giovp]

thanks all for the discussion/clarification. Imho the current specs are too strict requiring parameters that are purely for visualization purposes, hindering the flexibility of storing the channel information in the metadata slot. I opened #192 to discuss this further.

Will close this for now, as my understanding from this discussion is that the channel name will not be affected by the new coordinate systems and transformations proposal, and will live in the omero metadata until a new metadata spec proposal will be pushed forward.