[email protected] package is broken: h5cc -show does not work
#159691
Comments
|
$ h5cc -show
clang -I/opt/homebrew/opt/libaec/include -L/opt/homebrew/Cellar/[email protected]/1.10.11/lib /opt/homebrew/Cellar/[email protected]/1.10.11/lib/libhdf5_hl.a /opt/homebrew/Cellar/[email protected]/1.10.11/lib/libhdf5.a -L/opt/homebrew/opt/libaec/lib -Wl,-ld_classic -lsz -lz -ldl -lm
$ h5cc -showconfig
SUMMARY OF THE HDF5 CONFIGURATION
=================================
General Information:
-------------------
HDF5 Version: 1.10.11
Configured on: Wed Sep 27 22:08:03 UTC 2023
Configured by: [email protected]
Host system: aarch64-apple-darwin23.0.0
Uname information: Darwin Ventura-arm64.local 23.0.0 Darwin Kernel Version 23.0.0: Thu Aug 17 21:24:15 PDT 2023; root:xnu-10002.1.11~3/RELEASE_ARM64_VMAPPLE arm64
Byte sex: little-endian
Installation point: /opt/homebrew/Cellar/[email protected]/1.10.11
Compiling Options:
------------------
Build Mode: production
Debugging Symbols: no
Asserts: no
Profiling: no
Optimization Level: high
Linking Options:
----------------
Libraries: static, shared
Statically Linked Executables:
LDFLAGS: -Wl,-ld_classic
H5_LDFLAGS:
AM_LDFLAGS: -L/opt/homebrew/opt/libaec/lib
Extra libraries: -lsz -lz -ldl -lm
Archiver: ar
AR_FLAGS: cr
Ranlib: ranlib
Languages:
----------
C: yes
C Compiler: clang ( Apple clang version 15.0.0 )
CPPFLAGS:
H5_CPPFLAGS: -DNDEBUG -UH5_DEBUG_API
AM_CPPFLAGS: -I/opt/homebrew/opt/libaec/include
C Flags:
H5 C Flags: -std=c99 -Wall -Warray-bounds -Wcast-qual -Wconversion -Wdouble-promotion -Wextra -Wformat=2 -Wframe-larger-than=16384 -Wimplicit-fallthrough -Wnull-dereference -Wunused-const-variable -Wwrite-strings -Wpedantic -Wvolatile-register-var -Wno-c++-compat -Wbad-function-cast -Wimplicit-function-declaration -Wincompatible-pointer-types -Wmissing-declarations -Wpacked -Wshadow -Wswitch -Wno-error=incompatible-pointer-types-discards-qualifiers -Wunused-function -Wunused-variable -Wunused-parameter -Wcast-align -Wformat -Wno-missing-noreturn -O3 @H5_ECFLAGS@
AM C Flags:
Shared C Library: yes
Static C Library: yes
Fortran: yes
Fortran Compiler: /opt/homebrew/opt/gcc/bin/gfortran ( GNU Fortran (Homebrew GCC 13.2.0) 13.2.0)
Fortran Flags:
H5 Fortran Flags: -std=f2008 -Waliasing -Wall -Wcharacter-truncation -Wextra -Wimplicit-interface -Wsurprising -Wunderflow -pedantic -Wintrinsics-std -Wimplicit-procedure -Wreal-q-constant -Wfunction-elimination -Wrealloc-lhs -Wrealloc-lhs-all -Wno-c-binding-type -Winteger-division -Wfrontend-loop-interchange -fdiagnostics-urls=never -fno-diagnostics-color -s -Wno-unused-dummy-argument -Wno-array-temporaries -O3
AM Fortran Flags:
Shared Fortran Library: yes
Static Fortran Library: yes
Module Directory: ${includedir}
C++: yes
C++ Compiler: clang++ ( Apple clang version 15.0.0 )
C++ Flags:
H5 C++ Flags: -std=c++98 -Wall -Warray-bounds -Wcast-qual -Wconversion -Wdouble-promotion -Wextra -Wformat=2 -Wframe-larger-than=16384 -Wimplicit-fallthrough -Wnull-dereference -Wunused-const-variable -Wwrite-strings -Wpedantic -Wvolatile-register-var -Wno-c++-compat -Wbad-function-cast -Wimplicit-function-declaration -Wincompatible-pointer-types -Wmissing-declarations -Wpacked -Wshadow -Wswitch -Wno-error=incompatible-pointer-types-discards-qualifiers -Wunused-function -Wunused-variable -Wunused-parameter -Wcast-align -Wformat -O3 @H5_ECXXFLAGS@
AM C++ Flags: -DOLD_HEADER_FILENAME -DHDF_NO_NAMESPACE -DNO_STATIC_CAST
Shared C++ Library: yes
Static C++ Library: yes
Java: no
Features:
---------
Parallel HDF5: no
Parallel Filtered Dataset Writes: no
Large Parallel I/O: no
High-level library: yes
Build HDF5 Tests: yes
Build HDF5 Tools: yes
Threadsafety: no
Default API mapping: v110
With deprecated public symbols: yes
I/O filters (external): deflate(zlib),szip(encoder)
MPE: no
Direct VFD: no
Mirror VFD: no
(Read-Only) S3 VFD: no
(Read-Only) HDFS VFD: no
Packages w/ extra debug output: none
API tracing: no
Using memory checker: no
Memory allocation sanity checks: no
Function stack tracing: no
Use file locking: best-effort
Strict file format checks: no
Optimization instrumentation: no |
|
Could you report this upstream? I don't think brew's modifications have much of an impact on this. |
|
It may be a side effect of moving to CMake build system.
CMake was requested in #157078 though there may be some issues #157078 (comment) Please check with upstream on feature compatibility and if there is specific build recommendations. |
|
Yes, the CMake build of HDF5 is completely broken. I have built it from source dozens of times over the years, and it has never worked. Homebrew should build it with autotools. |
Indeed, this was caused by #159165, which should be reverted. |
|
We'll consider reverting if we can't get fixes submitted upstream. The goal for Homebrew is to fix any bugs we encounter upstream, not revert and wait until they magically fix themselves. Please help by reporting the issue upstream and, if at all possible, thinking of some patches for the problem. |
It has been broken for literally decades. There is almost no chance of upstream fixing this. |
|
We won't know for sure until we try. All it might take is a bug report. We got some help on the CMake PR 🤷 |
|
Seems like it's expected behaviour and they don't want people relying on
Upstream don't agree CMake is broken as they're planning to retire autotools builds: https://www.hdfgroup.org/2022/11/can-we-remove-the-autotools/ |
|
Unfortunately, CMake itself still relies on the use of So keeping this as-is breaks all builds of projects that use CMake's built-in functionality to link against HDF5. That is why all Linux distributions (AFAIK) build HDF5 via autotools. |
That code only runs if it can't find the HDF5 CMake config which should be available: https://github.com/Kitware/CMake/blob/e315d7cb19f019b91e68f7d3ca5720f294c70bab/Modules/FindHDF5.cmake#L509-L592. Unless you're disabling that with If that config is indeed missing then that is definitely a significant bug. |
It's not available. (We are not setting that CMake variable.) This problem caused all of our macOS CI builds to fail on GitHub actions runners. |
|
Thanks for the info - that's actually a bug on our side which I'll fix. |
This is also a chicken-and-egg problem as I discussed at #159165 (comment) The tl;dr here is that for autotools builds of hdf5, the only way to detect hdf5 is with For third-party projects which build with literally any other build system on the planet, the cmake config files are totally useless and don't exist. So the only way to reliably detect hdf5 is to run Third-party projects which don't use cmake, cannot stop using h5cc -show, because linux distros etc. all ship autotools builds of hdf5, where only h5cc is available. They cannot start using pkg-config, because pkg-config is only provided by cmake, which no one packages. Users are stuck in a holding pattern. And hdf5 claims to want to provide pkg-config files in their autotools builds, but has major commitment issues (this is a general theme of hdf5 when it comes to build systems, sadly). As I commented on the other ticket:
So, erm.
Congratulations, you've officially discovered the problem that all the other distros already knew about and were unable to get fixed. Experiment concluded, homebrew has successfully done its part in the FOSS ecosystem by discovering bugs in order to report them and think of patches for the problem. My advice: now that you've discovered this, you've done your part and can revert it now. In parallel, you can go try to communicate with hdf5 upstream to fix said bugs. Things which are needed:
Allow projects to migrate from h5cc to pkg-config, on their own schedule that is NOT tied to "which build system was hdf5 built with". If hdf5 cannot make this work, then it seems unreasonable to break existing projects in the wild over it? |
|
The ticket has been incorrectly closed. The fact that Whether or not cmake config files are available does not ameliorate the problem for users of build systems other than cmake, which currently run |
This isn't entirely true given some Linux distros do package a pkg-config file. I've checked and some distros ship a pkg-config file:
+ probably some others I've missed. A couple of the above do use CMake builds to achieve this. Homebrew previously didn't because it stuck with the autotools build while some others used a blend of Autotools+CMake or potentially custom patched the gaps. We've received bug reports because we didn't use the modern pkg-config/CMake system. There is no superior path like you claim. For example, some things like the h5py bindings only support pkg-config and do not support |
The issue the original reporter described is now fixed, which was incompatibility with the CMake's FindHDF5 system due to an error on our part. |
|
Actually project('hdf5test', 'c')
h5c = dependency('hdf5', language: 'c')The Meson build system
Version: 1.3.1
Source dir: /private/tmp
Build dir: /private/tmp/build
Build type: native build
Project name: hdf5test
Project version: undefined
C compiler for the host machine: cc (clang 15.0.0 "Apple clang version 15.0.0 (clang-1500.1.0.2.5)")
C linker for the host machine: cc ld64 1022.1
Host machine cpu family: aarch64
Host machine cpu: aarch64
Found pkg-config: YES (/opt/homebrew/bin/pkg-config) 0.29.2
Run-time dependency HDF5 for c found: YES 1.14.3
Build targets in project: 0So that brings the known breakages back down to zero again, but if there's anything I've missed here then I'm happy to help out. I agree with you in that upstream really should have handled this better. |
|
Minimal repro case: project(test)
set(HDF5_USE_STATIC_LIBRARIES ON)
find_package(HDF5 REQUIRED)
add_executable(test test.c)
target_link_libraries(test HDF5::HDF5)// test.c
#include <hdf5.h>
int main() {
H5Fopen("", 0, 0);
}Enabling Also note that if I have |
|
Thank you for the repro! Looks like there was a change on our side I forgot to push - try Tested your example now and it seems to work with the latest build. |
|
Yep that's done the trick. Thanks for the fix! |
Sure. Debian builds with autotools, and also installs a set of pkg-config files they wrote from scratch. They often do that for projects that don't provide pkg-config files at all, which is not particularly collaborative and people cannot assume they exist... Alpine and Arch build and install with autotools, and also run cmake to generate pkg-config files only. I mentioned this somewhere I think. My opinion on Nix can be best stated as "no comment", but I was indeed unaware that FreeBSD and spack used cmake to build. :/
Meson does support both h5cc and pkg-config. Meson uses dependency provider "methods" which are selectable, so using For meson, I had proposed a meson patch on the 17th, which I'm about to land, that causes config-tool detection to abort with a "notfound" error when it detects that h5cc was produced via cmake. Since meson tests each dependency provider in CI to check that it works, we need both this and to annotate the test as "expected-fail" on macOS, since it... fails with homebrew hdf5. I still think the correct solution here is to install a functional h5cc... |
|
This seems to be broken on amd64 macos monterey, version 1.14.3 rebuild 3. Sample logs from a GitHub actions run: https://github.com/microsoft/yardl/actions/runs/7633454119/job/20795750850 The "original" rebuild 0 worked. |
|
Can confirm its broken, but in an entirely different way for me. amd64 on monterey works fine, according to this bit from the build artifacts of https://github.com/welch-lab/RcppPlanc/actions/runs/7919769437/job/21621294614#logs However, it DOESN'T work on arm64 sonoma (https://github.com/welch-lab/RcppPlanc/actions/runs/7919769437/job/21621294872) |
Edit: This was caused by #159165. That PR should be reverted.
brew gist-logs <formula>link ORbrew configANDbrew doctoroutputVerification
brew doctoroutput" saysYour system is ready to brew.and am still able to reproduce my issue.brew updateand am still able to reproduce my issue.brew doctorand that did not fix my problem.What were you trying to do (and why)?
hdf5providesh5cc, which is a command-line utility that is used by build systems (e.g., CMake) to determine what compiler flags to use to link with hdf5.This is done with
h5cc -show(https://manpages.debian.org/testing/hdf5-helpers/h5cc.1.en.html#show).What happened (include all command output)?
What did you expect to happen?
It should list the compiler flags needed to link with hdf5.
Note that
h5cc -showconfigworks, but this cannot be parsed by build systems:Step-by-step reproduction instructions (by running
brewcommands)The text was updated successfully, but these errors were encountered: