Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NETCDF3 endian error in python 3.12 on windows server #1373

Closed
veenstrajelmer opened this issue Oct 11, 2024 · 7 comments
Closed

NETCDF3 endian error in python 3.12 on windows server #1373

veenstrajelmer opened this issue Oct 11, 2024 · 7 comments

Comments

@veenstrajelmer
Copy link

veenstrajelmer commented Oct 11, 2024

  • netcdf4 version: 1.7.1.post2
  • env: Windows server 2022, Python 3.12.7

Error report
In a poetry environment with numpy 2.0.2 and netcdf4 1.7.1.post2 I get the following error in my test suite on github: "RuntimeError: only endian='native' allowed for NETCDF3 files, got 'big' (variable 'network', group '/')". This also happens for other variables, e.g. "RuntimeError: only endian='native' allowed for NETCDF3 files, got 'big' (variable 'mesh2d', group '/')".

The environment details and tracebacks are available here:
https://github.com/Deltares/HYDROLIB-core/actions/runs/11290766627/job/31403293222

Would this be something that will be solved with netcdf4 1.7.2 (or a new 1.7.2 post release like mentioned in #1370) or is this issue not on the radar yet?

Additional information
What I tried:

  • forcing NETCDF4_CLASSIC instead of NETCDF3_CLASSIC upon writing files only made three of the 21 tests pass. The rest is related to reading mostly and passing NETCDF4_CLASSIC did not change anything there. I would also like to prevent hardcoding that in my code anyway.
  • What makes it complex is that I cannot reproduce this on my local Windows 10 machine in the same poetry environment (via the lockfile) and the same python version (3.12.7).
  • when enforcing numpy 2.1.2 for python 3.12 (the same version as in your github tests), the errors still occur. I tried this since I have seem ValueError: numpy.dtype size changed when numpy and netcdf4 versions did not match up (Investigate numpy 2.0.0 support Deltares/HYDROLIB-core#657)
@chrishavlin
Copy link

Would this be something that will be solved with netcdf4 1.7.2 (or a new 1.7.2 post release like mentioned in #1370) or is this issue not on the radar yet?

@veenstrajelmer Looks like there is an unreleased fix already: #1355

@veenstrajelmer
Copy link
Author

Ok that is amazing. In that case, I am very much looking forward to a new release. When is that expected?

@chrishavlin
Copy link

Not sure if the question was directed to me, but I'm not part of the netcdf4 dev team, just someone else who had the same error and was looking through linked issues and related pull requests :) So I couldn't tell you when the next netcdf4-python release is planned for.

@veenstrajelmer
Copy link
Author

Fair enough, I was not paying attention. @jswhit could you please link this issue to PR #1355 (and close the issue) if that fix indeed resolves the problem in this issue? Also, now for several fixes, I am very much looking forward to a new release. When is that expected?

@jswhit
Copy link
Collaborator

jswhit commented Oct 19, 2024

Hope to create a new release this week

@jswhit
Copy link
Collaborator

jswhit commented Oct 22, 2024

there's a new release up on pypi. Feel free to re-open this issue if the problem persists.

@jswhit jswhit closed this as completed Oct 22, 2024
@veenstrajelmer
Copy link
Author

This issue was resolved with the 1.7.2 release, thanks a lot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants