Made equalized attributes in concatenated cubes consistent across runs

[schlunma]

Description

This PR fixes the ordering attributes for concatenated cubes from multiple source (e.g., multiple experiments). If values are equal across cubes, this value is not changed. If values differ, they are converted to strings, sorted, and joined by a delimiter (default: |). Example: exp: historical|ssp585.

Closes #1779

Link to documentation:

• https://esmvaltool--1783.org.readthedocs.build/projects/ESMValCore/en/1783/api/esmvalcore.iris_helpers.html#esmvalcore.iris_helpers.merge_cube_attributes
• https://esmvaltool--1783.org.readthedocs.build/projects/ESMValCore/en/1783/api/esmvalcore.typing.html#esmvalcore.typing.NetCDFAttr

---

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🧪 The new functionality is relevant and scientifically sound
• 🛠 This pull request has a descriptive title and labels
• 🛠 Code is written according to the code quality guidelines
• 🧪 and 🛠 Documentation is available
• 🛠 Unit tests have been added
• 🛠 Changes are backward compatible
• 🛠 Any changed dependencies have been added or removed correctly
• 🛠 The list of authors is up to date
• 🛠 All checks below this pull request were successful

---

To help with the number pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[codecov]

Codecov Report

Merging #1783 (559af1b) into main (684e7ea) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #1783   +/-   ##
=======================================
  Coverage   92.10%   92.11%           
=======================================
  Files         234      234           
  Lines       12133    12144   +11     
=======================================
+ Hits        11175    11186   +11     
  Misses        958      958           

Impacted Files	Coverage Δ
esmvalcore/iris_helpers.py	100.00% <100.00%> (ø)
esmvalcore/preprocessor/_io.py	84.10% <100.00%> (-0.64%)	⬇️
esmvalcore/typing.py	100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[zklaus]

I like this functionality, but I have three main comments. First, the typical delimiter used in CF is simply " ", i.e. a space, which is not great for many reasons, but still worth considering. It may create problems for some variable values, but for the typical experiment id and variant label, etc., this would be fine and consistent with CF. Second, the name equalize clearly alludes to Iris' own iris.util.equalise_attributes. Differences in spelling aside (fine, IMHO), this does follow different semantics. A different name might be better (join_attributes? merge_attributes?). Third, due to the proximity to the Iris functionality and the ongoing work on making concatenation and merging more forgiving in Iris, I think we should at least ping the Iris team and see if later on they might be interested in upstreaming this, either in the concatenation itself or perhaps via the iris.common.resolve functionality.

[schlunma]

Thanks for your input @zklaus!

First, the typical delimiter used in CF is simply " ", i.e. a space, which is not great for many reasons, but still worth considering. It may create problems for some variable values, but for the typical experiment id and variant label, etc., this would be fine and consistent with CF.

Fine for me, I don't really have a strong opinion on this. I will change that.

Second, the name equalize clearly alludes to Iris' own iris.util.equalise_attributes. Differences in spelling aside (fine, IMHO), this does follow different semantics. A different name might be better (join_attributes? merge_attributes?).

Yes, that definitely makes sense. I will adapt the name.

Third, due to the proximity to the Iris functionality and the ongoing work on making concatenation and merging more forgiving in Iris, I think we should at least ping the Iris team and see if later on they might be interested in upstreaming this, either in the concatenation itself or perhaps via the iris.common.resolve functionality.

Good idea! I will open an issue and ask if this feature is something that might be interesting for them. Let's wait for their response until we merge this to avoid having to deprecate this in the next release 😄

[schlunma]

See SciTools/iris#5050

[bouweandela]

Nice work @schlunma! I left some comments on how the code could be further improved, but it looks fine as it is already.

[schlunma]

I wonder if we also should include this in v2.8. Looks like there won't be a solution in iris anytime soon, and this will be a further steps towards reproducible results.

[valeriupredoi]

Good idea! I will open an issue and ask if this feature is something that might be interesting for them. Let's wait for their response until we merge this to avoid having to deprecate this in the next release

You still want to do this @schlunma ? Otherwise I can merge this unless you are still waiting for an OK from @zklaus 🍺

[schlunma]

I opened the issue here SciTools/iris#5050, but I think we can merge it, does not look like we get an iris solution anytime soon