We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Zarr 2.11.0 made a change where chunks with data equal to the fill value are no longer written to disk by default. I would like to override the default, but this isn't possible using xarray's to_zarr.
to_zarr
No response
import numpy as np import xarray as xr ds = xr.Dataset({"x": (("d1"), np.full((2,), np.nan))}) ds.to_zarr("/tmp/ds.zarr", encoding={"x": {"write_empty_chunks":True}})
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/tom/miniconda/envs/sgkit-3.8/lib/python3.8/site-packages/xarray/core/dataset.py", line 2036, in to_zarr return to_zarr( File "/Users/tom/miniconda/envs/sgkit-3.8/lib/python3.8/site-packages/xarray/backends/api.py", line 1431, in to_zarr dump_to_store(dataset, zstore, writer, encoding=encoding) File "/Users/tom/miniconda/envs/sgkit-3.8/lib/python3.8/site-packages/xarray/backends/api.py", line 1119, in dump_to_store store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims) File "/Users/tom/miniconda/envs/sgkit-3.8/lib/python3.8/site-packages/xarray/backends/zarr.py", line 534, in store self.set_variables( File "/Users/tom/miniconda/envs/sgkit-3.8/lib/python3.8/site-packages/xarray/backends/zarr.py", line 581, in set_variables encoding = extract_zarr_variable_encoding( File "/Users/tom/miniconda/envs/sgkit-3.8/lib/python3.8/site-packages/xarray/backends/zarr.py", line 220, in extract_zarr_variable_encoding raise ValueError( ValueError: unexpected encoding parameters for zarr backend: ['write_empty_chunks']
commit: None python: 3.8.12 (default, Oct 12 2021, 06:23:56) [Clang 10.0.0 ] python-bits: 64 OS: Darwin OS-release: 19.6.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: ('en_GB', 'UTF-8') libhdf5: None libnetcdf: None
xarray: 2022.3.0 pandas: 1.3.5 numpy: 1.21.5 scipy: 1.7.3 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.0 cftime: None nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2022.01.0 distributed: 2022.01.0 matplotlib: 3.5.1 cartopy: None seaborn: 0.11.2 numbagg: None fsspec: 2022.01.0 cupy: None pint: None sparse: None setuptools: 58.0.4 pip: 21.2.4 conda: None pytest: 6.2.5 IPython: 8.0.1 sphinx: 4.2.0
The text was updated successfully, but these errors were encountered:
Successfully merging a pull request may close this issue.
What happened?
Zarr 2.11.0 made a change where chunks with data equal to the fill value are no longer written to disk by default. I would like to override the default, but this isn't possible using xarray's
to_zarr
.What did you expect to happen?
No response
Minimal Complete Verifiable Example
Relevant log output
Anything else we need to know?
No response
Environment
INSTALLED VERSIONS
commit: None
python: 3.8.12 (default, Oct 12 2021, 06:23:56)
[Clang 10.0.0 ]
python-bits: 64
OS: Darwin
OS-release: 19.6.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8
LOCALE: ('en_GB', 'UTF-8')
libhdf5: None
libnetcdf: None
xarray: 2022.3.0
pandas: 1.3.5
numpy: 1.21.5
scipy: 1.7.3
netCDF4: None
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: 2.11.0
cftime: None
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: 2022.01.0
distributed: 2022.01.0
matplotlib: 3.5.1
cartopy: None
seaborn: 0.11.2
numbagg: None
fsspec: 2022.01.0
cupy: None
pint: None
sparse: None
setuptools: 58.0.4
pip: 21.2.4
conda: None
pytest: 6.2.5
IPython: 8.0.1
sphinx: 4.2.0
The text was updated successfully, but these errors were encountered: