1.4.9: build fails

[kloczek]

Source tree is configured with:

%meson \
        -D backtrace=false \
        -D build_programs=true \
        -D build_tests=true \
        -D debug_level=0 \
        -D lz4=disabled \
        -D lzma=disabled \
        -D multi_thread=enabled \
        -D static_runtime=false \
        -D zlib=disabled \
        %{nil}

And it fails with:

[tkloczko@barrel x86_64-redhat-linux-gnu]$ ninja
[1/4] Linking target lib/libzstd.so.1.4.9
FAILED: lib/libzstd.so.1.4.9
/usr/bin/gcc  -o lib/libzstd.so.1.4.9 lib/libzstd.so.1.4.9.p/.._.._.._lib_common_entropy_common.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_common_fse_decompress.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_common_threading.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_common_pool.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_common_zstd_common.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_common_error_private.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_common_xxhash.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_compress_hist.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_compress_fse_compress.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_compress_huf_compress.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_compress_zstd_compress.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_compress_zstd_compress_literals.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_compress_zstd_compress_sequences.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_compress_zstd_compress_superblock.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_compress_zstdmt_compress.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_compress_zstd_fast.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_compress_zstd_double_fast.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_compress_zstd_lazy.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_compress_zstd_opt.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_compress_zstd_ldm.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_decompress_huf_decompress.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_decompress_zstd_decompress.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_decompress_zstd_decompress_block.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_decompress_zstd_ddict.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_dictBuilder_cover.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_dictBuilder_fastcover.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_dictBuilder_divsufsort.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_dictBuilder_zdict.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_deprecated_zbuff_common.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_deprecated_zbuff_compress.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_deprecated_zbuff_decompress.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_legacy_zstd_v05.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_legacy_zstd_v06.c.o lib/libzstd.so.1.4.9.p/.._.._.._lib_legacy_zstd_v07.c.o -Wl,--as-needed -Wl,--no-undefined -shared -fPIC -Wl,--start-group -Wl,-soname,libzstd.so.1 -Wl,--end-group -Wl,-z,relro -Wl,--as-needed -Wl,--gc-sections -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -flto=auto -flto-partition=none -fuse-linker-plugin -pthread
/usr/bin/ld: /usr/bin/ld: DWARF error: invalid abstract instance DIE ref
/tmp/cc3BMINj.lto.o: in function `ZSTD_CCtx_trace':
<artificial>:(.text+0x63aee): undefined reference to `ZSTD_trace_compress_end'
/usr/bin/ld: /tmp/cc3BMINj.lto.o: in function `ZSTD_compressBegin_internal':
<artificial>:(.text+0x6d83a): undefined reference to `ZSTD_trace_compress_begin'
/usr/bin/ld: /tmp/cc3BMINj.lto.o: in function `ZSTD_CCtx_init_compressStream2':
<artificial>:(.text+0x6f500): undefined reference to `ZSTD_trace_compress_begin'
/usr/bin/ld: /tmp/cc3BMINj.lto.o: in function `ZSTD_compress_advanced_internal':
<artificial>:(.text+0x72972): undefined reference to `ZSTD_trace_compress_begin'
/usr/bin/ld: /tmp/cc3BMINj.lto.o: in function `ZSTD_decompressBegin':
<artificial>:(.text+0x76159): undefined reference to `ZSTD_trace_decompress_begin'
/usr/bin/ld: /tmp/cc3BMINj.lto.o: in function `ZSTD_DCtx_trace_end.part.0':
<artificial>:(.text+0x78a12): undefined reference to `ZSTD_trace_decompress_end'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

[kloczek]

I'm using gcc 11.0,1 and here is set of env variables which I'm injecting into meson env:

$ rpm -E %set_build_flags

CFLAGS="-O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none";
CXXFLAGS="-O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none";
FFLAGS="-O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -I/usr/lib64/gfortran/modules";
FCFLAGS="-O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -I/usr/lib64/gfortran/modules";
LDFLAGS="-Wl,-z,relro -Wl,--as-needed -Wl,--gc-sections -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -flto=auto -flto-partition=none -fuse-linker-plugin";
CC="/usr/bin/gcc"; CXX="/usr/bin/g++"; FC="/usr/bin/gfortran";
AR="/usr/bin/gcc-ar"; NM="/usr/bin/gcc-nm"; RANLIB="/usr/bin/gcc-ranlib";
export CFLAGS CXXFLAGS FFLAGS FCFLAGS LDFLAGS CC CXX FC AR NM RANLIB;

[Cyan4973]

Seems like meson recipe is missing zstd_trace.c where these weak symbols are provided a default (empty) implementation.

[kloczek]

Looks like patch below fixes the issue:

--- a/build/meson/lib/meson.build~      2021-03-02 22:20:57.000000000 +0000
+++ b/build/meson/lib/meson.build       2021-03-03 21:56:00.058700361 +0000
@@ -22,6 +22,7 @@
   join_paths(zstd_rootdir, 'lib/common/threading.c'),
   join_paths(zstd_rootdir, 'lib/common/pool.c'),
   join_paths(zstd_rootdir, 'lib/common/zstd_common.c'),
+  join_paths(zstd_rootdir, 'lib/common/zstd_trace.c'),
   join_paths(zstd_rootdir, 'lib/common/error_private.c'),
   join_paths(zstd_rootdir, 'lib/common/xxhash.c'),
   join_paths(zstd_rootdir, 'lib/compress/hist.c'),

BTW: looks like zstd meson does not provide any test units using stuff from tests/.

+ /usr/bin/meson test -C x86_64-redhat-linux-gnu --num-processes 48 --print-errorlogs
No tests defined.

[kloczek]

BTW: looks like zstd meson does not provide any test units using stuff from tests/.

Sorry was wrong. -D bin_tests=true enables test suite.
I'm not sure is it right to keep that option disabled by default :P

[Cyan4973]

I vaguely remember that we used to have a meson build test in CI,
but it started to fail repeatedly for some external reason,
something related to ninja (wrong version, or impossibility to update, or whatever).
Well, anyway, this led to its disabling, as it was polluting the logs, making it harder to determine when other things were failing.

Sadly, no one uses meson here, so we have no skill, nor time to develop them, so the script is "community maintained".
I guess if someone can make the test work again, it could come back in CI.

[kloczek]

Just tested test suite and all looks good :)

+ /usr/bin/meson test -C x86_64-redhat-linux-gnu --num-processes 48 --print-errorlogs
ninja: Entering directory `/home/tkloczko/rpmbuild/BUILD/zstd-1.4.9/build/meson/x86_64-redhat-linux-gnu'
ninja: no work to do.
 1/11 test-invalidDictionaries         OK                0.03s
 2/11 test-zstream-2                   OK                0.06s
 3/11 test-poolTests                   OK                0.39s
 4/11 test-longmatch                   OK                2.27s
 5/11 test-fullbench-2                 OK               15.85s
 6/11 test-fullbench-1                 OK               18.78s
 7/11 test-decodecorpus                OK               30.10s
 8/11 test-zstream-3                   OK               86.57s
 9/11 test-zstream-1                   OK               94.73s
10/11 valgrindTest                     OK              117.08s
11/11 test-zstd                        OK              520.84s


Ok:                 11
Expected Fail:      0
Fail:               0
Unexpected Pass:    0
Skipped:            0
Timeout:            0

Full log written to /home/tkloczko/rpmbuild/BUILD/zstd-1.4.9/build/meson/x86_64-redhat-linux-gnu/meson-logs/testlog.txt

Personally I preffer meson over cmake (IMO meson more declarative build descried of the build is way better than cmake functional one).
All what IMO it would be good to change is enable test suite by default.
I'm using meson with zstd since it landed in zstd tree and only now I've realised that nothing is running by %meson_test from my zstd.spec file on building zstd. package.

[kloczek]

Do you want me to submit fix as PR or you will do that? :P

[Cyan4973]

You are certainly welcome to submit a PR.
It's likely going to be faster.

[kloczek]

OK
I'll do this and if you will agree will do in PR two commits. On with build fix and second one with enable test suite by default.
Is that would be OK?

[Cyan4973]

sure!