Allow extraction of variables without time dimension

[aaschwanden]

Hi,

I'm running into issues when trying to extract variables like 'lat/lon', that don't have a temporal dimension :

  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/ocgis-1.3.1-py2.7.egg/ocgis/api/request/driver/nc.py", line 285, in _get_field_
    raise ValueError('Target variable must at least have temporal, row, and column dimensions.')
ValueError: Target variable must at least have temporal, row, and column dimensions.

Here the variable only has (y,x) dimensions.

For certain variables, a temporal dimension doesn't make much sense. Would it be possible to support extraction of these variables?

[bekozi]

This will be supported in the next release and is part of the work related to #374. You are correct that this is an unfortunate limitation. Using netCDF4-python one could add a dummy temporal dimension for the interim. Let me know if you'd like an example of this!

[aaschwanden]

Great, thanks, I'm delighted to hear that this will be supported in the future.
I understand I can use netcdf4python to add a time dimension. Here my files already have a time dimension, so I used something like:

ncap2 -O -s "cell_area_t[$time,$y,$x]=0.f; 'sz_idt=time.size(); for(*idt=0 ; idt<sz_idt ; idt++) {cell_area_t[idt,$y,$x]=cell_area;}" in.nc in.nc

to get the job done. However my time series has 1250 time steps and the file is 14GB (one of my smallest files), and this operation takes ~30min. Do you think netcdf4python would be faster?

The only time I can think of where I need one of the (y,x) variables in postprocessing is when calculating mass fluxes by multiplying with the cell_area, so I could probably do that operation during the model simulation and generate a new diagnostic instead of during postprocessing.

Anyway, keep up the good work!

[bekozi]

Do you think netcdf4python would be faster?

I'm not sure netcdf4python would be faster in this case - though I don't have much personal experience with NCO speeds. I suppose you could do a parallel Python script but that may end up being more trouble than it's worth.

I was thinking that only a single count time dimension would be needed to avoid the current ocgis limitation. I doubt I understand your full requirements, however.

I could probably do that operation during the model simulation and generate a new diagnostic instead of during postprocessing.

Probably a good idea as you will save on some data wrangling.

Anyway, keep up the good work!

Thanks for the well wishes. 😄

[bekozi]

This functionality will be available in v2.x and is available in the https://github.com/NCPP/ocgis/tree/v-2.0.0.dev1 branch now.

@aaschwanden If you get a change to test, please pass along any feedback!

[aaschwanden]

Tested it; works for me! Many thanks for your efforts.

[bekozi]

Excellent. Thanks!