Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Missing variables when getting the "DatasetBuildError: key present and new value is different" warning #139

Open
pdruzhinin opened this issue May 14, 2020 · 0 comments
Open

Missing variables when getting the "DatasetBuildError: key present and new value is different" warning #139

pdruzhinin opened this issue May 14, 2020 · 0 comments

Comments

@pdruzhinin
Copy link

pdruzhinin commented May 14, 2020 •
edited
Loading

Hello,

I am getting the same error message as the one described in #66 using cfgrib 0.9.8.1 attempting to open ECMWF weather forecasts.

ds = cfgrib.open_dataset('/dbfs/mnt/ecmwf/D1D05060000050600011')

Traceback (most recent call last):
  File "/databricks/python/lib/python3.7/site-packages/cfgrib/dataset.py", line 592, in build_dataset_components
    dict_merge(variables, coord_vars)
  File "/databricks/python/lib/python3.7/site-packages/cfgrib/dataset.py", line 528, in dict_merge
    "key=%r value=%r new_value=%r" % (key, master[key], value)
cfgrib.dataset.DatasetBuildError: key present and new value is different: key='depthBelowLandLayer' value=Variable(dimensions=(), data=100) new_value=Variable(dimensions=(), data=7)
skipping variable: paramId==40 shortName='swvl2'
Traceback (most recent call last):
  File "/databricks/python/lib/python3.7/site-packages/cfgrib/dataset.py", line 592, in build_dataset_components
    dict_merge(variables, coord_vars)
  File "/databricks/python/lib/python3.7/site-packages/cfgrib/dataset.py", line 528, in dict_merge
    "key=%r value=%r new_value=%r" % (key, master[key], value)
cfgrib.dataset.DatasetBuildError: key present and new value is different: key='depthBelowLandLayer' value=Variable(dimensions=(), data=100) new_value=Variable(dimensions=(), data=7)
skipping variable: paramId==41 shortName='swvl3'
Traceback (most recent call last):
  File "/databricks/python/lib/python3.7/site-packages/cfgrib/dataset.py", line 592, in build_dataset_components
    dict_merge(variables, coord_vars)
  File "/databricks/python/lib/python3.7/site-packages/cfgrib/dataset.py", line 528, in dict_merge
    "key=%r value=%r new_value=%r" % (key, master[key], value)
cfgrib.dataset.DatasetBuildError: key present and new value is different: key='depthBelowLandLayer' value=Variable(dimensions=(), data=100) new_value=Variable(dimensions=(), data=28)
skipping variable: paramId==183 shortName='stl3'
Traceback (most recent call last):
  File "/databricks/python/lib/python3.7/site-packages/cfgrib/dataset.py", line 592, in build_dataset_components
    dict_merge(variables, coord_vars)
  File "/databricks/python/lib/python3.7/site-packages/cfgrib/dataset.py", line 528, in dict_merge
    "key=%r value=%r new_value=%r" % (key, master[key], value)
cfgrib.dataset.DatasetBuildError: key present and new value is different: key='depthBelowLandLayer' value=Variable(dimensions=(), data=100) new_value=Variable(dimensions=(), data=28)
skipping variable: paramId==139 shortName='stl1'
Traceback (most recent call last):
  File "/databricks/python/lib/python3.7/site-packages/cfgrib/dataset.py", line 592, in build_dataset_components
    dict_merge(variables, coord_vars)
  File "/databricks/python/lib/python3.7/site-packages/cfgrib/dataset.py", line 528, in dict_merge
    "key=%r value=%r new_value=%r" % (key, master[key], value)
cfgrib.dataset.DatasetBuildError: key present and new value is different: key='depthBelowLandLayer' value=Variable(dimensions=(), data=100) new_value=Variable(dimensions=(), data=0)
skipping variable: paramId==39 shortName='swvl1'
Traceback (most recent call last):
  File "/databricks/python/lib/python3.7/site-packages/cfgrib/dataset.py", line 592, in build_dataset_components
    dict_merge(variables, coord_vars)
  File "/databricks/python/lib/python3.7/site-packages/cfgrib/dataset.py", line 528, in dict_merge
    "key=%r value=%r new_value=%r" % (key, master[key], value)
cfgrib.dataset.DatasetBuildError: key present and new value is different: key='depthBelowLandLayer' value=Variable(dimensions=(), data=100) new_value=Variable(dimensions=(), data=0)

ds = cfgrib.open_datasets('/dbfs/mnt/ecmwf/D1D05060000050600011')

 Dimensions:              (latitude: 721, longitude: 1440)
 Coordinates:
     number               int64 0
     time                 datetime64[ns] 2020-05-06
     step                 timedelta64[ns] 00:00:00
     depthBelowLandLayer  int64 0
   * latitude             (latitude) float64 90.0 89.75 89.5 ... -89.75 -90.0
   * longitude            (longitude) float64 -180.0 -179.8 ... 179.5 179.8
     valid_time           datetime64[ns] 2020-05-06
 Data variables:
     swvl1                (latitude, longitude) float32 ...
     stl1                 (latitude, longitude) float32 ...
 Attributes:
     GRIB_edition:            1
     GRIB_centre:             ecmf
     GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
     GRIB_subCentre:          0
     Conventions:             CF-1.7
     institution:             European Centre for Medium-Range Weather Forecasts,
 <xarray.Dataset>
 Dimensions:              (latitude: 721, longitude: 1440)
 Coordinates:
     number               int64 0
     time                 datetime64[ns] 2020-05-06
     step                 timedelta64[ns] 00:00:00
     depthBelowLandLayer  int64 7
   * latitude             (latitude) float64 90.0 89.75 89.5 ... -89.75 -90.0
   * longitude            (longitude) float64 -180.0 -179.8 ... 179.5 179.8
     valid_time           datetime64[ns] 2020-05-06
 Data variables:
     swvl2                (latitude, longitude) float32 ...
     stl2                 (latitude, longitude) float32 ...
 Attributes:
     GRIB_edition:            1
     GRIB_centre:             ecmf
     GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
     GRIB_subCentre:          0
     Conventions:             CF-1.7
     institution:             European Centre for Medium-Range Weather Forecasts,
 <xarray.Dataset>
 Dimensions:              (latitude: 721, longitude: 1440)
 Coordinates:
     number               int64 0
     time                 datetime64[ns] 2020-05-06
     step                 timedelta64[ns] 00:00:00
     depthBelowLandLayer  int64 28
   * latitude             (latitude) float64 90.0 89.75 89.5 ... -89.75 -90.0
   * longitude            (longitude) float64 -180.0 -179.8 ... 179.5 179.8
     valid_time           datetime64[ns] 2020-05-06
 Data variables:
     swvl3                (latitude, longitude) float32 ...
     stl3                 (latitude, longitude) float32 ...
 Attributes:
     GRIB_edition:            1
     GRIB_centre:             ecmf
     GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
     GRIB_subCentre:          0
     Conventions:             CF-1.7
     institution:             European Centre for Medium-Range Weather Forecasts,
 <xarray.Dataset>
 Dimensions:              (latitude: 721, longitude: 1440)
 Coordinates:
     number               int64 0
     time                 datetime64[ns] 2020-05-06
     step                 timedelta64[ns] 00:00:00
     depthBelowLandLayer  int64 100
   * latitude             (latitude) float64 90.0 89.75 89.5 ... -89.75 -90.0
   * longitude            (longitude) float64 -180.0 -179.8 ... 179.5 179.8
     valid_time           datetime64[ns] 2020-05-06
 Data variables:
     swvl4                (latitude, longitude) float32 ...
     stl4                 (latitude, longitude) float32 ...
 Attributes:
     GRIB_edition:            1
     GRIB_centre:             ecmf
     GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
     GRIB_subCentre:          0
     Conventions:             CF-1.7
     institution:             European Centre for Medium-Range Weather Forecasts,
 <xarray.Dataset>
 Dimensions:        (isobaricInhPa: 20, latitude: 721, longitude: 1440)
 Coordinates:
     number         int64 0
     time           datetime64[ns] 2020-05-06
     step           timedelta64[ns] 00:00:00
   * isobaricInhPa  (isobaricInhPa) int64 1000 950 925 900 850 ... 70 50 30 20 10
   * latitude       (latitude) float64 90.0 89.75 89.5 ... -89.5 -89.75 -90.0
   * longitude      (longitude) float64 -180.0 -179.8 -179.5 ... 179.5 179.8
     valid_time     datetime64[ns] 2020-05-06
 Data variables:
     t              (isobaricInhPa, latitude, longitude) float32 ...
     u              (isobaricInhPa, latitude, longitude) float32 ...
     v              (isobaricInhPa, latitude, longitude) float32 ...
     q              (isobaricInhPa, latitude, longitude) float32 ...
     gh             (isobaricInhPa, latitude, longitude) float32 ...
     r              (isobaricInhPa, latitude, longitude) float32 ...
 Attributes:
     GRIB_edition:            1
     GRIB_centre:             ecmf
     GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
     GRIB_subCentre:          0
     Conventions:             CF-1.7
     institution:             European Centre for Medium-Range Weather Forecasts,
 <xarray.Dataset>
 Dimensions:     (latitude: 721, longitude: 1440)
 Coordinates:
     number      int64 0
     time        datetime64[ns] 2020-05-06
     step        timedelta64[ns] 00:00:00
     surface     int64 0
   * latitude    (latitude) float64 90.0 89.75 89.5 89.25 ... -89.5 -89.75 -90.0
   * longitude   (longitude) float64 -180.0 -179.8 -179.5 ... 179.2 179.5 179.8
     valid_time  datetime64[ns] 2020-05-06
 Data variables:
     siconc      (latitude, longitude) float32 ...
     rsn         (latitude, longitude) float32 ...
     sst         (latitude, longitude) float32 ...
     cape        (latitude, longitude) float32 ...
     sp          (latitude, longitude) float32 ...
     tcwv        (latitude, longitude) float32 ...
     sd          (latitude, longitude) float32 ...
     lsp         (latitude, longitude) float32 ...
     cp          (latitude, longitude) float32 ...
     sf          (latitude, longitude) float32 ...
     msl         (latitude, longitude) float32 ...
     tcc         (latitude, longitude) float32 ...
     u10         (latitude, longitude) float32 ...
     v10         (latitude, longitude) float32 ...
     t2m         (latitude, longitude) float32 ...
     d2m         (latitude, longitude) float32 ...
     ssrd        (latitude, longitude) float32 ...
     lcc         (latitude, longitude) float32 ...
     mcc         (latitude, longitude) float32 ...
     hcc         (latitude, longitude) float32 ...
     tco3        (latitude, longitude) float32 ...
     tp          (latitude, longitude) float32 ...
     skt         (latitude, longitude) float32 ...
     fdir        (latitude, longitude) float32 ...
     deg0l       (latitude, longitude) float32 ...
     u100        (latitude, longitude) float32 ...
     v100        (latitude, longitude) float32 ...
 Attributes:
     GRIB_edition:            1
     GRIB_centre:             ecmf
     GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
     GRIB_subCentre:          0
     Conventions:             CF-1.7
     institution:             European Centre for Medium-Range Weather Forecasts]

I can share one of the grib files if required.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@pdruzhinin