Opening ATL08

[rwegener2]

Hi folks,

I'm trying to open ATL08 data, and I'm getting an error about conflicting dimension sizes. I've tried a few different granules and I get the same error each time.

Code to download the file (runs well):

import icepyx as ipx

short_name = 'ATL08'
spatial_extent = [-99.8, 21, -99.0, 21.7]
date_range = ['2019-06-11','2019-06-11']
region = ipx.Query(short_name, spatial_extent, date_range)
region.download_granules('./downloads_ATL08')

Reading in the file (produces the error)

pattern = "processed_ATL{product:2}_{datetime:%Y%m%d%H%M%S}_{rgt:4}{cycle:2}{orbitsegment:2}_{version:3}_{revision:2}.h5"
reader = ipx.Read('./downloads_ATL08', "ATL08", pattern)
reader.vars.append(beam_list=['gt1l', 'gt3r'], var_list=['h_canopy', "latitude", "longitude"])
reader.load()

The .load() line returns ValueError: cannot reindex or align along dimension 'delta_time' because of conflicting dimension sizes: {1, 26} (full stack trace below).

Is this error from something I have done or is there a bug for this product? I have been reading about the Variables class, since my only guess right now is that I've asked for an odd variable combination and that is causing the dimension error. Thanks so much for any insight!

Full stack trace

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[36], line 1
----> 1 reader.load()

File /srv/conda/envs/notebook/lib/python3.10/site-packages/icepyx/core/read.py:659, in Read.load(self)
    652 # DevNote: I'd originally hoped to rely on intake-xarray in order to not have to iterate through the files myself,
    653 # by providing a generalized url/source in building the catalog.
    654 # However, this led to errors when I tried to combine two identical datasets because the single dimension was equal.
    655 # In these situations, xarray recommends manually controlling the merge/concat process yourself.
    656 # While unlikely to be a broad issue, I've heard of multiple matching timestamps causing issues for combining multiple IS2 datasets.
    657 for file in self._filelist:
    658     all_dss.append(
--> 659         self._build_single_file_dataset(file, groups_list)
    660     )  # wanted_groups, vgrp.keys()))
    662 if len(all_dss) == 1:
    663     return all_dss[0]

File /srv/conda/envs/notebook/lib/python3.10/site-packages/icepyx/core/read.py:837, in Read._build_single_file_dataset(self, file, groups_list)
    833                     ds = Read._combine_nested_vars(
    834                         ds, sub_ds, grp_path2, wanted_dict
    835                     )
    836                     wanted_groups_list.remove(grp_path2)
--> 837             is2ds = is2ds.merge(ds, join="outer", combine_attrs="no_conflicts")
    839 return is2ds

File /srv/conda/envs/notebook/lib/python3.10/site-packages/xarray/core/dataset.py:5630, in Dataset.merge(self, other, overwrite_vars, compat, join, fill_value, combine_attrs)
   5627 from xarray.core.dataarray import DataArray
   5629 other = other.to_dataset() if isinstance(other, DataArray) else other
-> 5630 merge_result = dataset_merge_method(
   5631     self,
   5632     other,
   5633     overwrite_vars=overwrite_vars,
   5634     compat=compat,
   5635     join=join,
   5636     fill_value=fill_value,
   5637     combine_attrs=combine_attrs,
   5638 )
   5639 return self._replace(**merge_result._asdict())

File /srv/conda/envs/notebook/lib/python3.10/site-packages/xarray/core/merge.py:1038, in dataset_merge_method(dataset, other, overwrite_vars, compat, join, fill_value, combine_attrs)
   1035     objs = [dataset, other_no_overwrite, other_overwrite]
   1036     priority_arg = 2
-> 1038 return merge_core(
   1039     objs,
   1040     compat,
   1041     join,
   1042     priority_arg=priority_arg,
   1043     fill_value=fill_value,
   1044     combine_attrs=combine_attrs,
   1045 )

File /srv/conda/envs/notebook/lib/python3.10/site-packages/xarray/core/merge.py:710, in merge_core(objects, compat, join, combine_attrs, priority_arg, explicit_coords, indexes, fill_value, skip_align_args)
    707 skip_align_objs = [(pos, objects.pop(pos)) for pos in skip_align_args]
    709 coerced = coerce_pandas_values(objects)
--> 710 aligned = deep_align(
    711     coerced, join=join, copy=False, indexes=indexes, fill_value=fill_value
    712 )
    714 for pos, obj in skip_align_objs:
    715     aligned.insert(pos, obj)

File /srv/conda/envs/notebook/lib/python3.10/site-packages/xarray/core/alignment.py:847, in deep_align(objects, join, copy, indexes, exclude, raise_on_invalid, fill_value)
    844     else:
    845         out.append(variables)
--> 847 aligned = align(
    848     *targets,
    849     join=join,
    850     copy=copy,
    851     indexes=indexes,
    852     exclude=exclude,
    853     fill_value=fill_value,
    854 )
    856 for position, key, aligned_obj in zip(positions, keys, aligned):
    857     if key is no_key:

File /srv/conda/envs/notebook/lib/python3.10/site-packages/xarray/core/alignment.py:783, in align(join, copy, indexes, exclude, fill_value, *objects)
    587 """
    588 Given any number of Dataset and/or DataArray objects, returns new
    589 objects with aligned indexes and dimension sizes.
   (...)
    773 
    774 """
    775 aligner = Aligner(
    776     objects,
    777     join=join,
   (...)
    781     fill_value=fill_value,
    782 )
--> 783 aligner.align()
    784 return aligner.results

File /srv/conda/envs/notebook/lib/python3.10/site-packages/xarray/core/alignment.py:569, in Aligner.align(self)
    567 self.assert_no_index_conflict()
    568 self.align_indexes()
--> 569 self.assert_unindexed_dim_sizes_equal()
    571 if self.join == "override":
    572     self.override_indexes()

File /srv/conda/envs/notebook/lib/python3.10/site-packages/xarray/core/alignment.py:470, in Aligner.assert_unindexed_dim_sizes_equal(self)
    468     add_err_msg = ""
    469 if len(sizes) > 1:
--> 470     raise ValueError(
    471         f"cannot reindex or align along dimension {dim!r} "
    472         f"because of conflicting dimension sizes: {sizes!r}" + add_err_msg
    473     )

ValueError: cannot reindex or align along dimension 'delta_time' because of conflicting dimension sizes: {1, 26}

[weiji14]

Yep, same bug as #466

[andypbarrett]

I do not think it is a product problem, I can open https://n5eil02u.ecs.nsidc.org/esir/5000004605258/229534395/ATL08_20190611103530_11330301_005_01.h5 using xarray. For both gt1l and gt3r.