recommended file extension for zarr-v3 stores (.zr3?)

[grlee77]

Currently the spec does not specify how a v3 store should be named. I am curious whether we should either encourage or enforce use of a specific suffix like .zr3 (currently used for the folder names for v3 cases in the zarr_implementations tests) or .zarr3.

If this was enforced, we could infer from a file path which version of the Zarr protocol the store was created with without having to provide a zarr_version=3 kwarg (or implementing some kind of inspection of the file hierarchy to try and autodetect the version).

Currently in the draft implementation of zarr-developers/zarr-python#898, no such autoinspection is attempted. If a function takes a store argument and a StoreV3 subclass is passed in, it is safe to assume version 3. Otherwise version 2 is assumed for backwards compatibility unless a zarr_version=3 kwarg is passed in.

[joshmoore]

Bringing this up informally to OME community members, I received a fairly strong response against changing the file-ending for Zarr V3. At the next OME community meeting, I will try to poll further.

[jakirkham]

Yeah the metadata already has a version in it even for v2. So should be able to figure this out without an extension.

That said, we don't have a file extension convention atm. Should we have one? Or is that to store specific (IOW is this thinking too much about filesystems)?

[joshmoore]

I don't think it's necessarily filesystem specific. If the metadata of the object storage presents anything name-like, users will likely (and I'd argue rightly) try to interpret them based on the string in order to do the "right thing". I think it would be better to have a proactive recommendation, but there's definitely a conversation to be had around enforcement.

seealso: bids-standard/bids-specification#1104 (comment)

[jstriebel]

For the metadata, we settled on the name zarr.json for v3, which contains a key format_version. Since zarr hierarchies can be openened at any level, I don't see where a file extension should be used for folders in file-stores. Perhaps this is different with zip stores or other single files representing a store. Is there something you would like to see in the core spec in the context of ZEP 1, @joshmoore @grlee77 @jakirkham?

[joshmoore]

Personally I see "the root of a Zarr fileset SHOULD end with .zarr" as a pretty good compromise.

[jstriebel]

Personally I see "the root of a Zarr fileset SHOULD end with .zarr" as a pretty good compromise.

Sounds good to me 👍 I'd rather add that as a recommendation, since having a file-suffix e.g. for folders might be strange, I remember @ryan-williams disliking it.

See commit 5db7a75 in #211, please feel free to propose changes there directly.