Support HyperSpy 2.0

[hakonanes]

HyperSpy 2.0 is slated for release around mid-September according to hyperspy/hyperspy#2996 (comment). What does it take for kikuchipy to be compatible with this release?

An example of kikuchipy's failing test suite run with HyperSpy 2.0: https://github.com/hyperspy/hyperspy-extensions-list/actions/runs/5563155043/job/15068218388.

Current incompatibilities that I'm aware of are related to HyperSpy's IO moving to RosettaSciIO, which we should use. Features that must be handled:

• Saving to HSPY/ZSPY and image formats.
• Internal use of the function overwrite_dataset() for overwriting. We can import this from the hspy plugin in HyperSpy 1.7.5, but not from the same plugin in RosettaSciIO 0.1.
• Use of rgb_tools. We can import this from HyperSpy 1.7.5, but not from RosettaSciIO 0.1.

Other incompatibilities:

• Our creation of markers from geometrical EBSD simulations may will not work with the new marker collections in Marker Collection hyperspy/hyperspy#3148

[hakonanes]

• Use of rgb_tools. We can import this from HyperSpy 1.7.5, but not from RosettaSciIO 0.1.

I have to correct myself. This is not part of the listed public API, but it can be imported as from rsciio.utils import rgb_tools.

• Internal use of the function overwrite_dataset() for overwriting. We can import this from the hspy plugin in HyperSpy 1.7.5, but not from the same plugin in RosettaSciIO 0.1.

This function, however, can only be imported as from rsciio.hspy._api import overwrite_dataset. Thus, it is not part of the public API nor can it be imported from a "public looking" module.

[CSSFrancis]

@hakonanes assuming that the markers PR gets merged that will also have to be changed.

I was planning on making a PR for that once everything starts to get fixed there.

It might be worthwhile to think if there is anything that better annotation might help with. I've been thinking about overlaying phase information on the navigator. That or it might be nice to make something like arrange multiple navigators using an abstraction of hyperspy/hyperspy#3140.

It might also be good to let hyperspy plot colored plots for navigators as well.

[hakonanes]

The use of markers must be changed as well, of course. I didn't think of this. Thanks. A PR with the necessary changes would be great!

We navigate EBSD signals using phase and orientation color maps in kikuchipy already (see e.g. our visualization tutorial). It is very useful. We have a convenience function for this, get_rgb_navigator(), which takes an array of shape (n rows, n columns, 3) and returns a suitable signal with rgb8 or rgb16 data type. This might be a good utility function to move to HyperSpy?

Linking multiple navigators would be useful, indeed. This might already be possible with the current events functionality? I haven't tried it. A convenience function would be useful, accepting a signal and multiple suitable arrays or signals to use as navigators.

[hakonanes]

This work will be pushed to a v0.10.0 release as the new release of PyEBSDIndex v0.2.0 breaks kikuchipy from conda-forge (not from PyPI). See the progress of that work at #652.

[CSSFrancis]

@hakonanes I was going to see about changing the marker implementation but I'm not sure I fully understand how the io plugins work in kikuchipy.

[hakonanes]

Thank you for showing interest in using HyperSpy 2.0 in kikuchipy! I'm almost done repurposing our IO plugins to use RosettaSciIO in #651. It might be easier to make kikuchipy compatible with HS 2.0 after that.

I don't think I'll finish that PR the coming week, though. I plan to use my spare time on orix instead. I'll get back to you once the PR is done!

[CSSFrancis]

@hakonanes any movement on this? It would be good to make a hyperspy-bundle release relatively soon.

[hakonanes]

None since mid December. I plan to finish this in the latter half of February. A minor release will be made soon after that. If the bundle can't wait, I guess releasing one without kikuchipy is the only option.

[CSSFrancis]

@hakonanes I don't think there is any major rush on the bundle, these things just take time. At the very latest it would be good to make a bundle release before the next hyperspy/hyperspy affiliated training session. (I'm not sure exactly when that is)

Let me know if you need some help! I realize that starting a new job etc. means that you might not have as much to time to devote to some of these things.

[hakonanes]

Let me know if you need some help!

Thank you, that is very much appreciated.

I realize that starting a new job etc. means that you might not have as much to time to devote to some of these things.

Yes, I naturally have to reduce my commitment to adding new features to softwares I help maintain. But I will help maintain them (discuss here, fix issues and the like).

[hakonanes]

I've restarted this work as I tried to do all compatibility changes at once in #651. Instead, I've created a hyperspy2-support branch in this repo. Smaller PRs towards v2 compatiblity goes into this branch. This is hopefully more manageable.