Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add xarray backend for Igor Pro files #60

Draft
wants to merge 37 commits into
base: main
Choose a base branch
from
Draft

Add xarray backend for Igor Pro files #60

wants to merge 37 commits into from

Conversation

kmnhan
Copy link
Owner

@kmnhan kmnhan commented Sep 4, 2024

Add a backend so that igor pro files (.pxp, .pxt, .ibw) can be opened with xr.open_dataarray and xr.open_dataset.

Links:
https://docs.xarray.dev/en/stable/internals/how-to-add-new-backend.html

  • Implement backend entry point
  • Change io functions to use xarray methods like xr.open_mfdataset
    After testing with dask, joblib-based parallelization seems to work better (at least for igor files)
  • Update documentation on working with igor
  • Update documentation with changes to data loader
  • Drop support returning list of items, it was prone to breaking anyways, replace with DataTree and close Integrate DataTree #29

`.pxt`, `.pxp`, and `.ibw` files can now be opened with xarray methods such as `xr.open_dataset` and `xr.open_dataarray`. See the updated user guide for more information.
@kmnhan kmnhan added enhancement New feature or request docs Documentation related labels Sep 4, 2024
@kmnhan kmnhan self-assigned this Sep 4, 2024
Copy link

codecov bot commented Sep 4, 2024

Codecov Report

Attention: Patch coverage is 83.37731% with 63 lines in your changes missing coverage. Please review.

Project coverage is 77.36%. Comparing base (6fcc4a8) to head (bb36298).

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/erlab/io/dataloader.py 79.33% 25 Missing ⚠️
src/erlab/plotting/annotations.py 22.22% 7 Missing ⚠️
src/erlab/io/__init__.py 25.00% 6 Missing ⚠️
src/erlab/io/igor.py 80.64% 6 Missing ⚠️
src/erlab/utils/formatting.py 90.76% 6 Missing ⚠️
src/erlab/io/plugins/merlin.py 77.77% 4 Missing ⚠️
src/erlab/io/plugins/ssrl52.py 0.00% 2 Missing ⚠️
src/erlab/accessors/general.py 0.00% 1 Missing ⚠️
src/erlab/accessors/kspace.py 83.33% 1 Missing ⚠️
src/erlab/interactive/imagetool/slicer.py 97.67% 1 Missing ⚠️
... and 4 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #60      +/-   ##
==========================================
+ Coverage   75.94%   77.36%   +1.41%     
==========================================
  Files          64       65       +1     
  Lines       10543    10667     +124     
==========================================
+ Hits         8007     8252     +245     
+ Misses       2536     2415     -121     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Calling `erlab.io.load_wave` and `erlab.io.load_experiment` is deprecated. When writing new code, use `xarray.load_dataarray` and `xarray.load_dataset` instead.
Whenever the `identify()` method failed to find any files, subclasses had to explicitly raise `FileNotFoundError`. This resulted in a lot of boilerplate code and ambiguous error messages. Now, all subclasses can just return `None` in `identify()` when no files are found. The appropriate error is automatically raised.
Memmapping seemed to interfere loading multiple files when called through ipywidgets.
@kmnhan kmnhan force-pushed the dev-io branch 5 times, most recently from 4141c42 to 08930bb Compare September 11, 2024 04:59
The previous behavior allowed passing `f_001.pxt` to load `f_001_S001.pxt`, `f_001_S002.pxt`... but this was confusing since there is no file named `f_001.pxt`. This commit disallows such input.
Validation checks will not warn about missing detector angle, allowing XPS measurements.
…loaders

Combining before renaming coords should be more straightforward
A new attribute named `formatters` and a new method `get_formatted_attr_or_coord` has been added to loaders. This allows custom per-attribute pretty-printing behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Documentation related enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate DataTree
1 participant