autotools not building DLL using msys2 and mingw64 on windows 10

[jonforums]

With the current configure.ac and src/Makefile.am, libtool fails to build libonig-5.dll when using a plain vanilla msys2 + mingw64 toolchain on windows 10 x64.

To build a usable oniguruma, I need to use sed , autoreconf -fi, make src dll, strip the libtool created dll, and finally copy the libonig.def file to the install bin/ dir as in my build helper https://github.com/jonforums/buildlets/blob/master/build_oniguruma.ps1#L45-L72

The problem is that the libonig_la_LDFLAGS does not include -no-undefined when the --build triplet (os portion) contains mingw*.

PCRE2 solves this problem with the following autotools config:

### file: Makefile.am
if WITH_PCRE2_8
libpcre2_8_la_LDFLAGS = $(EXTRA_LIBPCRE2_8_LDFLAGS)
endif # WITH_PCRE2_8

---

### file: configure.ac
# Platform specific issues
NO_UNDEFINED=
EXPORT_ALL_SYMBOLS=
case $host_os in
  cygwin* | mingw* )
    if test X"$enable_shared" = Xyes; then
      NO_UNDEFINED="-no-undefined"
      EXPORT_ALL_SYMBOLS="-Wl,--export-all-symbols"
    fi
    ;;
esac
...
EXTRA_LIBPCRE2_8_LDFLAGS="$EXTRA_LIBPCRE2_8_LDFLAGS \
  $NO_UNDEFINED -version-info libpcre2_8_version"

[kkos]

I'm not sure exactly what to do, but I made the change based on PCRE2 10.35.
I'll create a release candidate 3 later if this is acceptable.
I don't have a Windows environment, so I can't be sure.

[jonforums]

Later tonight I will download the latest from the master branch, try a plain vanilla build, and reply with results.

[jonforums]

No problems building master using the manual build shown below. A quick test of the dll seems to work.

$ cd staging/bin
$ perl -m'FFI::Platypus' -e '$ffi=FFI::Platypus->new(lib => q(libonig-5.dll), api => 1); $ffi->attach(q(onig_version), [], q(string)); printf qq(Onig version: %s\n), onig_version()'
Onig version: 6.9.6

But libtool is not building the .def when it does libtool: link: gcc -shared ... -o .libs/libonig-5.dll.

I'm not sure if there is another var like libonig_la_LDFLAGS that libtool uses when building the final dll via gcc -shared. If so, you could have configure.ac create a mingw* guarded var with -Wl,--output-def,libonig.def similar to how you are handling $(EXTRA_LIBONIG_LDFLAGS) for this issue.

build summary

## EXPORT_ALL_SYMBOLS looks to be unused in PCRE2's autoconf files
$ grep -nP EXPORT_ALL_SYMBOLS .\configure.ac .\Makefile.am
.\configure.ac:861:EXPORT_ALL_SYMBOLS=
.\configure.ac:866:      EXPORT_ALL_SYMBOLS="-Wl,--export-all-symbols"

---

$ curl.exe -L -O https://github.com/kkos/oniguruma/archive/master.zip

$ 7za x .\master.zip && cd oniguruma-master

$ gcc --version && make --version
gcc.exe (Rev4, Built by MSYS2 project) 10.2.0
...
GNU Make 4.3
Built for x86_64-pc-msys

$ sh -c "./config.sub $(sh -c './config.guess')"
x86_64-pc-msys

$ sh -c 'autoreconf -fvi'

$ sh -c './configure --prefix=R:/staging --build=x86_64-w64-mingw32' | tee build.log
...
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
...
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating test/Makefile
config.status: creating sample/Makefile
config.status: creating onig-config
config.status: creating src/config.h
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing default commands

$ sh -c 'make' | tee build.log -append
Making all in src
make[1]: Entering directory '/r/oniguruma-master/src'
make  all-am
make[2]: Entering directory '/r/oniguruma-master/src'
...
/bin/sh ../libtool  --tag=CC   --mode=link gcc -Wall   -g -O2 -no-undefined -version-info 6:0:1
 -o libonig.la -rpath R:/staging/lib regparse.lo regcomp.lo regexec.lo regenc.lo regerror.lo regext.lo regsyntax.lo regtrav.lo regversion.lo st.lo reggnu.lo  unicode.lo unicode_unfold_key.lo unicode_fold1_key.lo unicode_fold2_key.lo unicode_fold3_key.lo ascii.lo utf8.lo utf16_be.lo utf16_le.lo utf32_be.lo utf32_le.lo euc_jp.lo euc_jp_prop.lo sjis.lo sjis_prop.lo iso8859_1.lo iso8859_2.lo iso8859_3.lo iso8859_4.lo iso8859_5.lo iso8859_6.lo iso8859_7.lo iso8859_8.lo iso8859_9.lo iso8859_10.lo iso8859_11.lo iso8859_13.lo iso8859_14.lo iso8859_15.lo iso8859_16.lo euc_tw.lo
euc_kr.lo big5.lo gb18030.lo koi8_r.lo cp1251.lo onig_init.lo

libtool: link: gcc -shared  .libs/regparse.o .libs/regcomp.o .libs/regexec.o .libs/regenc.o .libs/regerror.o .libs/regext.o .libs/regsyntax.o .libs/regtrav.o .libs/regversion.o .libs/st.o .libs/reggnu.o .libs/unicode.o .libs/unicode_unfold_key.o .libs/unicode_fold1_key.o .libs/unicode_fold2_key.o .libs/unicode_fold3_key.o .libs/ascii.o .libs/utf8.o .libs/utf16_be.o .libs/utf16_le.o .libs/utf32_be.o .libs/utf32_le.o .libs/euc_jp.o .libs/euc_jp_prop.o .libs/sjis.o .libs/sjis_prop.o .libs/iso8859_1.o .libs/iso8859_2.o .libs/iso8859_3.o .libs/iso8859_4.o .libs/iso8859_5.o .libs/iso8859_6.o .libs/iso8859_7.o .libs/iso8859_8.o .libs/iso8859_9.o .libs/iso8859_10.o .libs/iso8859_11.o .libs/iso8859_13.o .libs/iso8859_14.o .libs/iso8859_15.o .libs/iso8859_16.o .libs/euc_tw.o .libs/euc_kr.o .libs/big5.o .libs/gb18030.o .libs/koi8_r.o .libs/cp1251.o .libs/onig_init.o    -g -O2   -o .libs/libonig-5.dll -Wl,--enable-auto-image-base -Xlinker --out-implib -Xlinker .libs/libonig.dll.a


$ sh -c "make install" | tee build.log -append

$ ll ../staging/bin, ../staging/lib

    Directory: R:\staging\bin

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-----          10/13/2020 11:19 PM        2058626 libonig-5.dll
-----          10/13/2020 11:19 PM           1380 onig-config

    Directory: R:\staging\lib

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d----          10/13/2020 11:19 PM                pkgconfig
-a---          10/13/2020 11:19 PM        2307750 libonig.a
-----          10/13/2020 11:19 PM         209568 libonig.dll.a
-----          10/13/2020 11:19 PM            897 libonig.la

[jonforums]

The compile and link stage now looks good including libtool building the dll via

libtool: link: gcc -shared  ...  -g -O2 -Wl,--output-def -Wl,libonig.def   -o .libs/libonig-5.dll -Wl,--enable-auto-image-base -Xlinker --out-implib -Xlinker .libs/libonig.dll.a

But make install did not copy libonig.def to ${install_dir}/lib or ${install_dir}/bin.

I thought the reason was because libtool built libonig.def in src while libonig-5.dll, libonig.a, libonig.dll.a, and libonig.la were built in src/.libs/. But when I hard coded .libs in configure.ac to

FIX_TO_LDFLAGS="-no-undefined -Wl,--output-def,.libs/libonig.def"

libtool built libonig.def in src/.libs/ alongside libonig-5.dll but make install still did not copy libonig.def to the install dir.

Libtool did not strip ligonig-5.dll.

The good news is that dll is usable

$ .\build\onig.exe
2020/10/14 18:46:33 Loading libonig-5.dll...
2020/10/14 18:46:33 Oniguruma version: 6.9.6

from go using this

package main

import (
	"bytes"
	"log"
	"syscall"
	"unsafe"

	"golang.org/x/sys/windows"
)

const (
	VERSION       = "0.5.0"
	ONIGURUMA_DLL = "libonig-5.dll"

	LOAD_LIBRARY_SEARCH_APPLICATION_DIR = 0x00000200
	LOAD_LIBRARY_SEARCH_SYSTEM32        = 0x00000800
)

func main() {
	log.Printf("Loading %s...\n", ONIGURUMA_DLL)
	h, err := windows.LoadLibraryEx(ONIGURUMA_DLL, 0,
		uintptr(LOAD_LIBRARY_SEARCH_APPLICATION_DIR))
	if err != nil {
		log.Fatalf("Unable to load `%s`: %s\n", ONIGURUMA_DLL, err)
	}
	defer windows.FreeLibrary(h)

	proc, err := windows.GetProcAddress(h, "onig_version")
	if err != nil {
		log.Fatal("Unable to find `onig_version`: ", err)
	}

	r, _, errno := syscall.Syscall(uintptr(proc), 0, 0, 0, 0)
	if errno != 0 {
		log.Fatal("Unable to call `onig_version`: ", error(errno))
	}

	b := (*[1 << 4]byte)(unsafe.Pointer(r))[:]
	log.Printf("Oniguruma version: %s\n", string(b[:bytes.IndexByte(b, 0)]))
}

[kkos]

Changed to install libonig.def in make install.
If you run "make install-strip", won't it be striped?

[jonforums]

619ae35 works great.

Both make install and make install-strip work as expected. Both bin/libonig-5.dll and lib/libonig.def are created correctly for the stripped and non-stripped scenarios.

[kkos]

Thank you for your help.
I was able to get release candidate 3 out.