netcdf4 conversion problem in simplify_network

[davide-f]

Checklist

• I am using the current main branch or the latest release. Please indicate.
• I am running on an up-to-date pypsa-earth environment. Update via conda env update -f envs/environment.yaml.

Describe the Bug

When executing the workflow on Namibia only, I am experiencing the PyPSA-related problem encountered here: PyPSA/PyPSA#204 (message already added).
The problem seems to be related to the presence of dc links in the country that leads the component "links_underground" to be a boolean value and that leads to conflicts.
However, other variables are boolean, so it seems something weird.

We need to address the DC-related stuff in a consistent way and that has strong implications into simplify_network

Error Message

INFO:__main__:Removing stubs
INFO:pypsa.io:Exported network elec_s.nc has lines, buses, loads, carriers, links, generators
Traceback (most recent call last):
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/davidef/.vscode-server/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/davidef/.vscode-server/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/davidef/.vscode-server/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/home/davidef/.vscode-server/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/davidef/.vscode-server/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/davidef/.vscode-server/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/data/davidef/git_world/pypsa-earth/scripts/simplify_network.py", line 553, in <module>
    n.export_to_netcdf(snakemake.output.network)
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/pypsa/io.py", line 665, in export_to_netcdf
    with ExporterNetCDF(path, least_significant_digit) as exporter:
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/pypsa/io.py", line 47, in __exit__
    self.finish()
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/pypsa/io.py", line 361, in finish
    self.ds.to_netcdf(self.path)
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/xarray/core/dataset.py", line 1901, in to_netcdf
    return to_netcdf(
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/xarray/backends/api.py", line 1072, in to_netcdf
    dump_to_store(
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/xarray/backends/api.py", line 1119, in dump_to_store
    store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/xarray/backends/common.py", line 265, in store
    self.set_variables(
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/xarray/backends/common.py", line 303, in set_variables
    target, source = self.prepare_variable(
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/xarray/backends/netCDF4_.py", line 464, in prepare_variable
    datatype = _get_datatype(
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/xarray/backends/netCDF4_.py", line 139, in _get_datatype
    return _nc4_dtype(var)
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/xarray/backends/netCDF4_.py", line 160, in _nc4_dtype
    raise ValueError(f"unsupported dtype for netCDF4 variable: {var.dtype}")
ValueError: unsupported dtype for netCDF4 variable: bool

@pz-max this issue seems to be one of the reasons why the run_scenario failed in many regions; this problem occurs when running the workflow on Namibia-only.

Environment attached
env.txt