-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Align FieldArray with Python xarray and netCDF data model #48
Comments
See PALEOtoolkit/PALEOmodel.jl#48 Initial test updating NamedDimension so that `coords` is just a vector of names of preferred coordinates. NB: shows that there are no tests for get_region in this package.
This will need changes in both PALEOboxes and PALEOmodel in order to update the abstractions used for grids, dimensions and coordinates, and more clearly define the data model that is used to store model output internally (PALEOmodel.OutputWriters.OutputMemory) and on disk (currently just a serialisation of the internal format using jld2) Currently the internal (and hence serialized external) format for model output uses the NamedDimension and AbstractGrid implementation. So the first priority has to be to implement a netCDF data format for storing model output on disk, so that the internal implementation can be updated without affecting existing work-in-progress that relies on saved output. Then the data model for grids/dimensions/coordinates and the representation of model output can be updated. One design choice that should then be reconsidered is how to represent grid information internally: via an abstract interface (as at present), vs an explicit lower-level data model (as netCDF does). Currently PALEOboxes grids implement an interface, and don't expose dimensions etc in a low level generic way. Instead there are three api calls:
|
See PALEOtoolkit#48 PALEOtoolkit/PALEOboxes.jl#85 - Generalize FieldArray - move get_region and coordinate manipulation code from PALEOboxes.jl - rename FixedCoord -> RecordCoord as now only used by FieldRecord (and should be used by OutputWriter) TODO this currently allows a model (only the black sea config tested) to run and store output, but the FieldRecord etc (and surely FieldArray and plotting) does not work. Stop at this point as this illustrates a review of the data model for internal data storage (at least) is needed. Which requires implementing a standardised stable output format on disk first.
The
PALEOmodel.FieldArray
is intended to work in the same way as the Pythonxarray
, which itself uses the data model from netCDFCF conventions
. Currently there are some key differences:The text was updated successfully, but these errors were encountered: