scion fluorescence

[isgiddy]

Describe the bug
Issue with dimensions when loading fluorescence data
dimension name = scicon_wlbb2fl_wlbb2fl_data_point

To Reproduce

Try load the data:

names = [
'wlbb2fl_FL2sig', # Fluorescence
'wlbb2fl_time'
]
ds_dict = gt.load.seaglider_basestation_netCDFs(os.path.join(datadir,filenames), names, return_merged=True, keep_global_attrs=False)

4. TypeError: Argument 'arg' has incorrect type (expected str, got numpy.str_)

Expected behavior
Load as per other variables.

Desktop (please complete the following information):

• OS: [e.g. iOS]

[lukegre]

Issy! 😃 Can you give the full error please. Very hard to solve these issues without the full callback. use the 'insert code' feature to add the callback so that it is readable

[isgiddy]

Here is the error:

names = [
    'wlbb2fl_time',
    'wlbb2fl_FL2sig' 
]

ds_dict = gt.load.seaglider_basestation_netCDFs(os.path.join(datadir,filenames), names,  return_merged=True, keep_global_attrs=False)`

`DIMENSION: wlbb2fl_data_point
{wlbb2fl_time}
100%|██████████| 487/487 [00:03<00:00, 138.49it/s]

DIMENSION: scicon_wlbb2fl_wlbb2fl_data_point
{wlbb2fl_FL2sig, wlbb2fl_time}
100%|██████████| 487/487 [00:02<00:00, 174.24it/s]
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
 in 
      4 ]
      5 
----> 6 ds_dict = gt.load.seaglider_basestation_netCDFs(os.path.join(datadir,filenames), names,  return_merged=True, keep_global_attrs=False)
      7 #dat2 = ds_dict['wlbb2fl_data_point']
      8 # interp chl to rest of datapoints

~/src/code/my_python/GliderTools/glidertools/load/seaglider.py in load_multiple_vars(files, variable_names, return_merged, verbose, keep_global_attrs, netcdf_attrs, keep_variable_attrs)
    448         # adding columns to dimension based dataframes one by one
    449         for col in df:
--> 450             data[dim_name][col] = df[col]
    451 
    452     ## MERGING DATA IF POSSIBLE

~/opt/anaconda3/lib/python3.7/site-packages/pandas/core/frame.py in __setitem__(self, key, value)
   2924 
   2925         # see if we can slice the rows
-> 2926         indexer = convert_to_index_sliceable(self, key)
   2927         if indexer is not None:
   2928             # either we have a slice or we have a string that can be converted

~/opt/anaconda3/lib/python3.7/site-packages/pandas/core/indexing.py in convert_to_index_sliceable(obj, key)
   2277         if idx._supports_partial_string_indexing:
   2278             try:
-> 2279                 return idx._get_string_slice(key)
   2280             except (KeyError, ValueError, NotImplementedError):
   2281                 return None

~/opt/anaconda3/lib/python3.7/site-packages/pandas/core/indexes/datetimes.py in _get_string_slice(self, key, use_lhs, use_rhs)
    776     def _get_string_slice(self, key: str, use_lhs: bool = True, use_rhs: bool = True):
    777         freq = getattr(self, "freqstr", getattr(self, "inferred_freq", None))
--> 778         _, parsed, reso = parsing.parse_time_string(key, freq)
    779         loc = self._partial_date_slice(reso, parsed, use_lhs=use_lhs, use_rhs=use_rhs)
    780         return loc

TypeError: Argument 'arg' has incorrect type (expected str, got numpy.str_)

The error lies in the dimension name, which doesn't have a corresponding time variable under the same dimension. Just for chl I downloaded the data separately and interpolated onto the ctd_data_point time sampling using the following method:

`def mergechl(datadir):
ls=os.listdir(datadir)
ls.sort()

return xr.concat([xr.open_dataset(os.path.join(datadir,filename))['wlbb2fl_FL2sig'] for filename in ls],dim='scicon_wlbb2fl_wlbb2fl_data_point')        

chl = mergechl(datadir)

names = [
'wlbb2fl_time',
]

ds_dict = gt.load.seaglider_basestation_netCDFs(os.path.join(datadir,filenames), names, return_merged=False, keep_global_attrs=False)

dat2 = ds_dict['wlbb2fl_data_point']

interp chl to rest of datapoints

chlInt=np.interp(dat.time_raw,dat2.wlbb2fl_time,chl)`

[lukegre]

@isgiddy Do you think there would be a way of implementing your solution in the code? Or would it be better to add this as an example in the documentation?