Unable to open GRIB2 with duplicate shortName fields

[Plantain]

I have a series of grib files with duplicate shortName fields for one parameter, with the distinguishing data being in the "name" field.
Opening these with cfgrib.open_datasets() seems to stomp on one of them and return only one of the them.
See attached, field(s) icaht
cb.grb2.gz
Can I workaround this with a filter by name somehow to still get both values out of the file with cfgrib?

[alexamici]

@Plantain you should be able to get the missing variable by passing filter_by_name={'name': 'long name of the lost variable'} (untested).

Please report back if it helps.

[Plantain]

Unfortunately it doesn't help, because the long names are the same as well. Actually, every single parameter that cfgrib exposes to filter on seems to be identical. The single parameter that does seem to differ is "typeOfFirstFixedSurface", so I've patched DATA_ATTRIBUTES_KEYS in cfgrib/dataset.py to allow me to filter by that. Which makes me wonder, why have the DATA_ATTRIBUTES_KEYS limitation at all, why not let users filter by whatever keys they please?

These particular GRIB2 files are published operationally by a national weather agency...

[alexamici]

@Plantain in the branch stable/0.9.7.x you can specify the read_keys backend_kwargs to read additional keys form the GRIB and then you are able to filter on them.

xr.open_dataset('cb.grb2', engine='cfgrib', backend_kwargs={'read_keys': ['typeOfFirstFixedSurface'], 'filter_by_keys': {'typeOfFirstFixedSurface': '3'}})                                                                              
Out[13]: 
<xarray.Dataset>
Dimensions:     (latitude: 721, longitude: 1440)
Coordinates:
    time        datetime64[ns] ...
    step        timedelta64[ns] ...
    level       int64 ...
  * latitude    (latitude) float64 -90.0 -89.75 -89.5 -89.25 ... 89.5 89.75 90.0
  * longitude   (longitude) float64 0.0 0.25 0.5 0.75 ... 359.2 359.5 359.8
    valid_time  datetime64[ns] ...
Data variables:
    icaht       (latitude, longitude) float32 ...
Attributes:
    GRIB_edition:            2
    GRIB_centre:             edzw
    GRIB_centreDescription:  Offenbach 
    GRIB_subCentre:          255
    Conventions:             CF-1.7
    institution:             Offenbach 
    history:                 2019-09-24T09:25:01 GRIB to CDM+CF via cfgrib-0....

In [14]: xr.open_dataset('cb.grb2', engine='cfgrib', backend_kwargs={'read_keys': ['typeOfFirstFixedSurface'], 'filter_by_keys': {'typeOfFirstFixedSurface': '2'}})                                                                              
Out[14]: 
<xarray.Dataset>
Dimensions:     (latitude: 721, longitude: 1440)
Coordinates:
    time        datetime64[ns] ...
    step        timedelta64[ns] ...
    level       int64 ...
  * latitude    (latitude) float64 -90.0 -89.75 -89.5 -89.25 ... 89.5 89.75 90.0
  * longitude   (longitude) float64 0.0 0.25 0.5 0.75 ... 359.2 359.5 359.8
    valid_time  datetime64[ns] ...
Data variables:
    icaht       (latitude, longitude) float32 ...
Attributes:
    GRIB_edition:            2
    GRIB_centre:             edzw
    GRIB_centreDescription:  Offenbach 
    GRIB_subCentre:          255
    Conventions:             CF-1.7
    institution:             Offenbach 
    history:                 2019-09-24T09:25:16 GRIB to CDM+CF via cfgrib-0....

This will be included in the upcoming 0.9.7.2 release.

[alexamici]

Ok, I just released it.