Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

autogen.sh fails #1343

Closed
jeffhammond opened this issue Jan 7, 2022 · 6 comments · Fixed by #1906 or #4253
Closed

autogen.sh fails #1343

jeffhammond opened this issue Jan 7, 2022 · 6 comments · Fixed by #1906 or #4253
Assignees
Labels
Component - Build CMake, Autotools Priority - 2. Medium ⏹ It would be nice to have this in the next release Type - Improvement Improvements that don't add a new feature or functionality
Milestone

Comments

@jeffhammond
Copy link

I do not understand why autogen.sh breaks on a very generic Linux machine.

autoreconf -fiv works just fine, so perhaps that should be documented as a superior option.

Error

jhammond@nuclear:~/HDF5/git$ ./autogen.sh -v

**************************
* HDF5 autogen.sh script *
**************************

Setting verbosity: high

dirname: missing operand
Try 'dirname --help' for more information.
Running trace script:
Finished processing HDF5 API calls:
	instrumented 0 API functions and 0 argument lists

Running error generation script:
Generating 'H5Epubgen.h'
Generating 'H5Einit.h'
Generating 'H5Eterm.h'
Generating 'H5Edefin.h'

Running API version generation script:
Generating 'src/H5version.h'

Running overflow macro generation script:
Generating 'H5overflow.h'

/libtoolize --copy --force
./autogen.sh: 254: /libtoolize: not found
./autogen.sh: 256: /libtoolize: not found

/usr/bin/libtoolize is present in the obvious default, so I do not know why it's not found.

Version

jhammond@nuclear:~/HDF5/git$ git log -n1
commit 9e6de287e7dee1f504f7c9cd7bfe32d8b349ee04 (HEAD -> develop, origin/develop, origin/HEAD)
Author: jhendersonHDF <[email protected]>
Date:   Tue Jan 4 11:00:55 2022 -0600

    Fix usage of several HDfprintf format specifiers after HDfprintf removal (#1324)
jhammond@nuclear:~/HDF5/git$ git status
On branch develop
Your branch is up to date with 'origin/develop'.

nothing to commit, working tree clean

Machine

jhammond@nuclear:~/HDF5/git$ uname -a
Linux nuclear 5.11.0-44-generic #48~20.04.2-Ubuntu SMP Tue Dec 14 15:36:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
@jhendersonHDF
Copy link
Collaborator

jhendersonHDF commented Jan 7, 2022

At first glance:

/libtoolize --copy --force

seems to be the issue. libtoolize certainly isn't going to be found there unless you have a strange system!

Looks like autogen.sh had a problem running dirname, which in turn causes LIBTOOL_DIR to be unset here. Is the output from which libtool on your system what you'd expect?

@jeffhammond
Copy link
Author

The issue is that libtool wasn't installed, because it comes from libtool-bin, whereas I only had the libtool package.

This happened to me on both of my Ubuntu 20 machines, both of which have the dependencies for all the HPC software I use, and I build e.g. LLVM and NWChem on them all the time, so they have the standard development packages installed.

I recommend more defense programming w.r.t. the following:

HDF5_LIBTOOL=$(which libtool)
..
LIBTOOL_DIR=`dirname ${HDF5_LIBTOOL}`
..
HDF5_LIBTOOLIZE="${LIBTOOL_DIR}/libtoolize"

@jhendersonHDF
Copy link
Collaborator

I recommend more defense programming w.r.t. the following

Agreed on this. It looks like we only use libtool to find the directory for libtoolize, so it may be fine to just do a which libtoolize instead. @derobins thoughts?

@derobins
Copy link
Member

It's been a long time since I looked at this code, but yeah - libtool-bin has to be installed. We can change it, we just have to be mindful of not breaking other platforms. I recall MacOS requiring hacks.

FWIW, we cobbled together a custom script instead of just running autoreconf because of older systems we had to support and the desire to hard-code the autotools versions we used to release. Those requirements may not be valid today, so it's probably worth revisiting that.

@derobins
Copy link
Member

We had to (temporarily) revert this change as it's breaking the NAG Fortran compiler and we have a release to get out the door.

@derobins derobins reopened this Jul 30, 2022
@derobins derobins assigned mattjala and derobins and unassigned mattjala May 4, 2023
@derobins derobins added Priority - 2. Medium ⏹ It would be nice to have this in the next release Component - Build CMake, Autotools Type - Improvement Improvements that don't add a new feature or functionality labels May 4, 2023
@derobins derobins added this to the 1.14.4 milestone Oct 17, 2023
@derobins
Copy link
Member

It's probably too late to get this into 1.14.3, but I'll try to merge #1906 back in and address the NAG Fortran issues for 1.14.4.

derobins added a commit to derobins/hdf5 that referenced this issue Mar 26, 2024
We previously tried removing the per-tool invocation of the Autotools
and instead simply invoked autoreconf (PR HDFGroup#1906). This was reverted
when it turned out that the NAG Fortran compiler had trouble with an
undecorated -shared linker flag.

It turns out that this is due to a bug in libtool 2.4.2 and earlier.
Since this version of libtool is over a decade old, we're un-reverting
the change. We've added a release note for anyone who has to build
from source on elderly platforms.

Fixes HDFGroup#1343
lrknox pushed a commit that referenced this issue Mar 27, 2024
We previously tried removing the per-tool invocation of the Autotools
and instead simply invoked autoreconf (PR #1906). This was reverted
when it turned out that the NAG Fortran compiler had trouble with an
undecorated -shared linker flag.

It turns out that this is due to a bug in libtool 2.4.2 and earlier.
Since this version of libtool is over a decade old, we're un-reverting
the change. We've added a release note for anyone who has to build
from source on elderly platforms.

Fixes #1343
lrknox pushed a commit to lrknox/hdf5 that referenced this issue Mar 29, 2024
We previously tried removing the per-tool invocation of the Autotools
and instead simply invoked autoreconf (PR HDFGroup#1906). This was reverted
when it turned out that the NAG Fortran compiler had trouble with an
undecorated -shared linker flag.

It turns out that this is due to a bug in libtool 2.4.2 and earlier.
Since this version of libtool is over a decade old, we're un-reverting
the change. We've added a release note for anyone who has to build
from source on elderly platforms.

Fixes HDFGroup#1343
lrknox added a commit that referenced this issue Mar 29, 2024
* Take user block into account when returning chunk addresses (#4236)

Both H5Dchunk_iter() and H5Dget_chunk_info(_by_coord)() did not take
the size of the user block into account when reporting addresses. Since
the #1 use of these functions is to root around in the file for the raw
data, this is kind of a problem.

Fixes GitHub issue #3003

* Fix a minor warning in h5test.c (#4242)

* Turn on -Werror for Java in GitHub -Werror workflows (#4243)

* Update Windows CI to not install ninja (#4230)

* Rework Fortran macros to use the proper code. (#4240)

* Correct reference copy for 16 API (#4244)

* Determine MPI LOGICAL during build, used in tests. (#4246)

* Skip userblock test in chunk_info.c for multi-file VFDs (#4249)

* Match generators with real cmake -G output on Windows (#4252)

* Add Julia GitHub Actions. (#4123)

* Re-revert to using autoreconf in autogen.sh (#4253)

We previously tried removing the per-tool invocation of the Autotools
and instead simply invoked autoreconf (PR #1906). This was reverted
when it turned out that the NAG Fortran compiler had trouble with an
undecorated -shared linker flag.

It turns out that this is due to a bug in libtool 2.4.2 and earlier.
Since this version of libtool is over a decade old, we're un-reverting
the change. We've added a release note for anyone who has to build
from source on elderly platforms.

Fixes #1343

* Rewrite H5T__path_find_real for clarity (#4225)

* Move conversion path free logic to helper function

* Add tgz extensions on names (#4255)

* Remove an error check regarding large cache objects (#4254)

* Remove an error check regarding large cache objects

In PR#4231 an assert() call was converted to a normal HDF5 error
check. It turns out that the original assert() was added by a
developer as a way of being alerted that large cache objects
existed instead of as a guard against incorrect behavior, making
it unnecessary in either debug or release builds.

The error check has been removed.

* Update RELEASE.txt

* File format security issues (#4234)

* Add job timeout to cygwin workflow (#4260)

* Replace user-define with user-defined (#4261)

* Improve the CMake clang -fsanitize=memory flags (#4267)

-fsanitize=memory is almost useless without
using -fsanitize-memory-track-origins=2 and we shoud probably add
-fno-optimize-sibling-calls as well.

* Add documentation (H5M) (#4259)

* Add documentation (H5P) (#4262)

* MPI type correction (#4268)

* corrected type for MPI_*_f2c APIs

* fixed return type of callback

* reset compilation flags of logical test program

* Clean up test/cmpd_dtransform.c (#4270)

* Clean up test/cmpd_dtransform.c

* Fix uninitialized memory warning from sanitizers
* FAIL_STACK_ERROR --> TEST_ERROR
* Emit output
* Delete test file when done

* Fix typo

* H5Fdelete() --> remove()

* Fix uninitialized memory issues in packet table (#4271)

* replace deprecated CMAKE_COMPILER_IS_GNU** (#4272)

* Prevent stack overflows in H5E__push_stack (#4264)

* Minor fixes after merge of file format security fixes (#4263)

* Update H5_IS_BUFFER_OVERFLOW to account for 'size' of 0

* Invert a few checks to avoid function call

* CHECK --> CHECK_PTR in tmisc.c (#4274)

* Add release note for CVE-2017-17507 (#4275)

* Update Cygwin installation guide (#4265)

* Addresses configuration fortran testing flags (#4276)

* turn warnings to errors in fortran configure test

* Intel fortran test fix

* Merge julia workflows into standard ci format (#4273)

* Fix range check in H5_addr_overlap (#4278)

When the H5_addr_overlap macro was updated to use H5_RANGE_OVERLAP,
it failed to take into account that H5_RANGE_OVERLAP expects the
range to be inclusive. This lead to an assertion failure in
H5MM_memcpy due to a memcpy operation on overlapping memory.
This has been fixed by subtracting 1 from the calculated high
bound values passed to H5_RANGE_OVERLAP

* Fix potential buffer read overflows in H5PB_read (#4279)

H5PB_read previously did not account for the fact that the size of the
read it's performing could overflow the page buffer pointer, depending
on the calculated offset for the read. This has been fixed by adjusting
the size of the read if it's determined that it would overflow the page.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component - Build CMake, Autotools Priority - 2. Medium ⏹ It would be nice to have this in the next release Type - Improvement Improvements that don't add a new feature or functionality
Projects
None yet
4 participants