Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compare with GODAD technique #13

Open
darothen opened this issue Jul 24, 2017 · 1 comment
Open

Compare with GODAD technique #13

darothen opened this issue Jul 24, 2017 · 1 comment

Comments

@darothen
Copy link
Owner

Charlie Zender documents a Group-Oriented Data Analysis and Distribution (GODAD) technique in the NCO documentation. This is actually really similar to what I'm trying to accomplish with this package, so there should be some opportunity for overlap and reinforcement.

@darothen
Copy link
Owner Author

The idea here is straightforward; every Case in an Experiment is encoded as a unique Group. Following the classic NetCDF data model, every Group can act as a unique NetCDF dataset. There are some rules about inheriting common coordinates/dimensions/attributes but we don't have to lean on them. The various levels of Cases in an Experiment thus would encode a Group path - just like the file hierarchy I've been toying about in the back of my mind. An Experiment-processed result could then serialize to an actual file system hierarchy on disk, or just a hierarchical NetCDF file with the Cases built into the Group paths.

This solves the issue of having non-aligned coordinates on the datasets read in by an Experiment; they don't have to be compared in many situations and can be treated differently. If everything further lives in dask territory, then all we really have to do is extend the xarray.Dataset interface to work on our Experiment containers by applying the operations to each Group/Case in our data cache.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant