-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compress::Raw::Zlib doesn't apply symbol versioning when building zlib-src #8
Comments
One way to do this would be to build libz.a, then link libz.a into the C::R::Zlib shared library. The linker would need |
Hey @monkburger thanks for the feedback. shout if I'm inferring too much here, but as things stand this is still a potential issue that hasn't been triggered anywhere that you know of? Getting back to the issue, can I make sure I understand it properly? This is only going to be an issue where Perl & C::R::Zlib are embedded in a larger application. Let's assume At load time the unversioned zlib symbols in C::R::Zlib will always be used in preference to the OS copy. That is a possible issue if the zlib source embedded in C::R::Zlib & the OS versions of zlib don't match. Is that about it? thanks |
Hi Paul. That is correct. I may have a demonstration of this behavior very soon - with a crash. I'll keep you posted. |
libperl links against distro zlib on AlmaLinux 8 because libnsl needs the distro zlib . As best I can tell,
|
Right now I cannot build blead on AlmaLinux 8. |
As best I can tell, a reasonable fix would be, instead of slurping all of zlib’s .o files directly into Zlib.so, create libz.a from the .o files, then slurp libz.a into Zlib.so, with the You’ll need to exclude platforms where the linker doesn’t support |
Is that because of this issue? |
^^ One-liner to determine if the linker supports the relevant option. |
Correct.
|
OK - means this isn't a hypothetical issue anymore |
I think there are two robust approaches to make this problem go away forever
|
Is it possible/feasible to include zlib as a github submodule? If so, you maybe could piggyback on top of zlib’s own build process. |
Rough proof-of-concept: https://github.com/FGasper/Compress-Raw-Zlib/tree/exclude_libs |
That's a possibility |
Maybe the static-linking approach could be confined to cases where the linker can |
Thanks! Will take a look |
A side note: zlib-src/zlib.h has this: Z_PREFIX Which does this, ie:
.. etc.. I compiled Compress::Raw::Zlib with -DZ_PREFIX and it seems to work (make test works), and the libz symbols are now prefixed with z_, ie:
|
Z_PREFIX might be the most expedient resolution for the immediate situation, then … assuming there isn’t some other zlib out there compiled with that option, of course. Long-term it would still seem safer not to export symbols at all. |
Completely forgot that zlib has that feature! This may be the path of least resistance to sort this out for now. |
Indeed.
It might be worthwhile to perhaps change the prefix to be crz_ instead of z_ ? (as a 'this is who I am') ? |
all libz (zlib-src/) symbols with z_, which should fix any symbol conflicts. XXX: It might be worthwhile to change the z_ prefix to something unique to C::R::Zlib, ie: crz_.
If you're going to do |
Was thinking along the same lines. I had thought that zlib would let you pick the prefix, but it would seem that Also need to look at the static link POC https://github.com/FGasper/Compress-Raw-Zlib/tree/exclude_libs from @FGasper in more detail. |
This enables -DZ_PREFIX, but prefixes external symbols with perl_crz_ instead of z_ (only when building w/ -DZ_PREFIX)
@pmqs I'm a little nervous about this fix not making 5.36 and Perl being unable to compile on RHEL/AlmaLinux 8. Are you ok with the PR? |
Hey @toddr Yes, I am ok with the change. Just haven't had the opportunity to spin a new release. Should be able to get to it by the weekend. Is there a cutoff date for 5.36? |
I'm being told: |
OK - I'll get sorted asap |
Just uploaded C:R:Zlib 2.104 to CPAN with fix from #11. Thanks everyone for chipping in with this |
It would be nice if BUILD_ZLIB is false (for building with system zlib) that the |
This commit synchs into blead versions 2.104 for CPAN distribution: Compress-Raw-Zlib 2.104 13 April 2022 Changes: * Merge pull request Perl#11 from monkburger/symbol_fix_2 Fri May 13 07:17:19 2022 +0100 64aea2d3f78946d7df4096eadfa0d7267f4439a5 * perl_crz -> Perl_crz Tue May 3 18:19:24 2022 +0000 20502e6c2eba8ddcad80b20574e840457c0cb369 * This is a slightly different way to fix pmqs/Compress-Raw-Zlib#8 Tue May 3 18:06:48 2022 +0000 d9cd27fb212da7455b6ba44729ca11bb441f3950 * add tests for crc32/adler32_combine Mon May 2 16:18:13 2022 +0100 dcfe9ef439790f1a4fae81cf3eac38cfeb848294
I think I may have run into what @pghmcfc mentioned (although it's possible I'm doing something wrong, too). After doing a portupgrade of this package on FreeBSD (which compiles it from source),
The munin plugin in question runs a Perl script using LWP::UserAgent, which may need to handle zlib-compressed streams. The distribution applies the following patch, setting BUILD_ZLIB to false:
I added the following to end of the port's
I'm not sure if that breaks it in some other way for other purposes, but it seems to have fixed my issue. |
For me the problem manifested just by running the test suite:
|
Issue reproduced - I'll get a fix together |
This commit synchs into blead versions 2.104 for CPAN distribution: Compress-Raw-Zlib 2.104 13 April 2022 Changes: * Merge pull request #11 from monkburger/symbol_fix_2 Fri May 13 07:17:19 2022 +0100 64aea2d3f78946d7df4096eadfa0d7267f4439a5 * perl_crz -> Perl_crz Tue May 3 18:19:24 2022 +0000 20502e6c2eba8ddcad80b20574e840457c0cb369 * This is a slightly different way to fix pmqs/Compress-Raw-Zlib#8 Tue May 3 18:06:48 2022 +0000 d9cd27fb212da7455b6ba44729ca11bb441f3950 * add tests for crc32/adler32_combine Mon May 2 16:18:13 2022 +0100 dcfe9ef439790f1a4fae81cf3eac38cfeb848294
Thank you! (The Changes file has 2.104 again, but I don't know if that has any material impact. ) |
Seems to be fixed. Thanks everyone! |
From ChangeLog: 2.202 27 June 2022 * Z_NULL should be 'UV' rather than 'PV' pmqs/Compress-Raw-Zlib#17 Sun Jun 26 22:02:04 2022 +0100 de28f0335d3d605d696b19d43fc48de42272455c 2.201 25 June 2022 * 2.021 Sat Jun 25 08:42:46 2022 +0100 85416cab509c18c5fa3f923de7b45b6c7c0f7a6f * 2.201 Sat Jun 25 08:39:26 2022 +0100 b3d63862b2ff4ac9d28e23be500c0d32ad69dd11 * More zlib-ng updates Thu Jun 23 22:42:13 2022 +0100 313f626425181702b5fc80af2b6ea7eed41d5a9d * Fix test count regression in t/07bufsize.t (Perl#16) Wed Jun 22 09:45:11 2022 +0100 98dc5b4a2b30c26752b6f686462b06b8db72a5e4 2.200 21 June 2022 * Added zlib-ng support pmqs/Compress-Raw-Zlib#9 * Only set Z_SOLO when building zlib sources * pmqs/Compress-Raw-Zlib#12 Tue Jun 7 10:13:00 2022 +0100 c44e0b732e214b7f77d42a3af6ae64ef944cee90 2.105 14 April 2022 * Add Compress::Raw::Zlib::VERSION to output Sat May 14 15:16:57 2022 +0100 3e22c93169a67986017f64d9a2e5085c417d8624 * Dump version info when running test harness Sat May 14 15:10:17 2022 +0100 ca9f33ba0323d0abc91a83800636f180b2b44162 * Fix use of ZLIB_INCLUDE/LIB Sat May 14 09:01:38 2022 +0100 8a7d4a97d7441b61a8a888342766419044fa5a33 * More fixes for BUILD_ZLIB off Sat May 14 08:54:04 2022 +0100 2d9650094dab90858ef58bfbda62f3bc60e159e4 * Add BUILD_ZLIB to the matrix Sat May 14 08:31:54 2022 +0100 b61b92fc9d06bf04f1adec337357ffbd39535901 * Merge branch 'master' of * https://github.com/pmqs/Compress-Raw-Zlib Sat May 14 08:27:14 2022 +0100 3ac7d0d3d45ae263402fab1ebb3835e2ae16c5a6 * Fix for BUILD_ZLIB disabled Sat May 14 08:25:34 2022 +0100 b0f04e37fb58a34ef01767ad16a8f63ca868eec6 * Add BUILD_ZLIB to the matrix Sat May 14 08:22:56 2022 +0100 aa8f5ff981c7305c995d4e2f798ae0d7d45866a5 2.104 13 April 2022 * Merge pull request #11 from monkburger/symbol_fix_2 Fri May 13 07:17:19 2022 +0100 64aea2d3f78946d7df4096eadfa0d7267f4439a5 * perl_crz -> Perl_crz Tue May 3 18:19:24 2022 +0000 20502e6c2eba8ddcad80b20574e840457c0cb369 * This is a slightly different way to fix * pmqs/Compress-Raw-Zlib#8 Tue May 3 18:06:48 2022 +0000 d9cd27fb212da7455b6ba44729ca11bb441f3950 * add tests for crc32/adler32_combine Mon May 2 16:18:13 2022 +0100 dcfe9ef439790f1a4fae81cf3eac38cfeb848294
From ChangeLog: 2.202 27 June 2022 * Z_NULL should be 'UV' rather than 'PV' pmqs/Compress-Raw-Zlib#17 Sun Jun 26 22:02:04 2022 +0100 de28f0335d3d605d696b19d43fc48de42272455c 2.201 25 June 2022 * 2.021 Sat Jun 25 08:42:46 2022 +0100 85416cab509c18c5fa3f923de7b45b6c7c0f7a6f * 2.201 Sat Jun 25 08:39:26 2022 +0100 b3d63862b2ff4ac9d28e23be500c0d32ad69dd11 * More zlib-ng updates Thu Jun 23 22:42:13 2022 +0100 313f626425181702b5fc80af2b6ea7eed41d5a9d * Fix test count regression in t/07bufsize.t (#16) Wed Jun 22 09:45:11 2022 +0100 98dc5b4a2b30c26752b6f686462b06b8db72a5e4 2.200 21 June 2022 * Added zlib-ng support pmqs/Compress-Raw-Zlib#9 * Only set Z_SOLO when building zlib sources * pmqs/Compress-Raw-Zlib#12 Tue Jun 7 10:13:00 2022 +0100 c44e0b732e214b7f77d42a3af6ae64ef944cee90 2.105 14 April 2022 * Add Compress::Raw::Zlib::VERSION to output Sat May 14 15:16:57 2022 +0100 3e22c93169a67986017f64d9a2e5085c417d8624 * Dump version info when running test harness Sat May 14 15:10:17 2022 +0100 ca9f33ba0323d0abc91a83800636f180b2b44162 * Fix use of ZLIB_INCLUDE/LIB Sat May 14 09:01:38 2022 +0100 8a7d4a97d7441b61a8a888342766419044fa5a33 * More fixes for BUILD_ZLIB off Sat May 14 08:54:04 2022 +0100 2d9650094dab90858ef58bfbda62f3bc60e159e4 * Add BUILD_ZLIB to the matrix Sat May 14 08:31:54 2022 +0100 b61b92fc9d06bf04f1adec337357ffbd39535901 * Merge branch 'master' of * https://github.com/pmqs/Compress-Raw-Zlib Sat May 14 08:27:14 2022 +0100 3ac7d0d3d45ae263402fab1ebb3835e2ae16c5a6 * Fix for BUILD_ZLIB disabled Sat May 14 08:25:34 2022 +0100 b0f04e37fb58a34ef01767ad16a8f63ca868eec6 * Add BUILD_ZLIB to the matrix Sat May 14 08:22:56 2022 +0100 aa8f5ff981c7305c995d4e2f798ae0d7d45866a5 2.104 13 April 2022 * Merge pull request #11 from monkburger/symbol_fix_2 Fri May 13 07:17:19 2022 +0100 64aea2d3f78946d7df4096eadfa0d7267f4439a5 * perl_crz -> Perl_crz Tue May 3 18:19:24 2022 +0000 20502e6c2eba8ddcad80b20574e840457c0cb369 * This is a slightly different way to fix * pmqs/Compress-Raw-Zlib#8 Tue May 3 18:06:48 2022 +0000 d9cd27fb212da7455b6ba44729ca11bb441f3950 * add tests for crc32/adler32_combine Mon May 2 16:18:13 2022 +0100 dcfe9ef439790f1a4fae81cf3eac38cfeb848294
This commit synchs into blead versions 2.104 for CPAN distribution: Compress-Raw-Zlib 2.104 13 April 2022 Changes: * Merge pull request Perl#11 from monkburger/symbol_fix_2 Fri May 13 07:17:19 2022 +0100 64aea2d3f78946d7df4096eadfa0d7267f4439a5 * perl_crz -> Perl_crz Tue May 3 18:19:24 2022 +0000 20502e6c2eba8ddcad80b20574e840457c0cb369 * This is a slightly different way to fix pmqs/Compress-Raw-Zlib#8 Tue May 3 18:06:48 2022 +0000 d9cd27fb212da7455b6ba44729ca11bb441f3950 * add tests for crc32/adler32_combine Mon May 2 16:18:13 2022 +0100 dcfe9ef439790f1a4fae81cf3eac38cfeb848294
From ChangeLog: 2.202 27 June 2022 * Z_NULL should be 'UV' rather than 'PV' pmqs/Compress-Raw-Zlib#17 Sun Jun 26 22:02:04 2022 +0100 de28f0335d3d605d696b19d43fc48de42272455c 2.201 25 June 2022 * 2.021 Sat Jun 25 08:42:46 2022 +0100 85416cab509c18c5fa3f923de7b45b6c7c0f7a6f * 2.201 Sat Jun 25 08:39:26 2022 +0100 b3d63862b2ff4ac9d28e23be500c0d32ad69dd11 * More zlib-ng updates Thu Jun 23 22:42:13 2022 +0100 313f626425181702b5fc80af2b6ea7eed41d5a9d * Fix test count regression in t/07bufsize.t (Perl#16) Wed Jun 22 09:45:11 2022 +0100 98dc5b4a2b30c26752b6f686462b06b8db72a5e4 2.200 21 June 2022 * Added zlib-ng support pmqs/Compress-Raw-Zlib#9 * Only set Z_SOLO when building zlib sources * pmqs/Compress-Raw-Zlib#12 Tue Jun 7 10:13:00 2022 +0100 c44e0b732e214b7f77d42a3af6ae64ef944cee90 2.105 14 April 2022 * Add Compress::Raw::Zlib::VERSION to output Sat May 14 15:16:57 2022 +0100 3e22c93169a67986017f64d9a2e5085c417d8624 * Dump version info when running test harness Sat May 14 15:10:17 2022 +0100 ca9f33ba0323d0abc91a83800636f180b2b44162 * Fix use of ZLIB_INCLUDE/LIB Sat May 14 09:01:38 2022 +0100 8a7d4a97d7441b61a8a888342766419044fa5a33 * More fixes for BUILD_ZLIB off Sat May 14 08:54:04 2022 +0100 2d9650094dab90858ef58bfbda62f3bc60e159e4 * Add BUILD_ZLIB to the matrix Sat May 14 08:31:54 2022 +0100 b61b92fc9d06bf04f1adec337357ffbd39535901 * Merge branch 'master' of * https://github.com/pmqs/Compress-Raw-Zlib Sat May 14 08:27:14 2022 +0100 3ac7d0d3d45ae263402fab1ebb3835e2ae16c5a6 * Fix for BUILD_ZLIB disabled Sat May 14 08:25:34 2022 +0100 b0f04e37fb58a34ef01767ad16a8f63ca868eec6 * Add BUILD_ZLIB to the matrix Sat May 14 08:22:56 2022 +0100 aa8f5ff981c7305c995d4e2f798ae0d7d45866a5 2.104 13 April 2022 * Merge pull request Perl#11 from monkburger/symbol_fix_2 Fri May 13 07:17:19 2022 +0100 64aea2d3f78946d7df4096eadfa0d7267f4439a5 * perl_crz -> Perl_crz Tue May 3 18:19:24 2022 +0000 20502e6c2eba8ddcad80b20574e840457c0cb369 * This is a slightly different way to fix * pmqs/Compress-Raw-Zlib#8 Tue May 3 18:06:48 2022 +0000 d9cd27fb212da7455b6ba44729ca11bb441f3950 * add tests for crc32/adler32_combine Mon May 2 16:18:13 2022 +0100 dcfe9ef439790f1a4fae81cf3eac38cfeb848294
There's a rare chance that when using the C:R:Z XS module under Linux (possibly others, like Solaris), there may be a symbol collision depending on your backend environment (perlcc, mod_perl, or other modules).
Zlib.so exports these functions, globally, without any symbol versions (via
readelf -s blib/arch/auto/Compress/Raw/Zlib/Zlib.so | egrep flate
)On ELF based systems (Linux, etc), unversioned symbols preempt versioned ones, so you could run into issues such as what was described in this bug report with CryptX, as well as these upstream reports relating to symbol versions, as well as this
He's what LD_DEBUG=all shows on some test code:
foo.1358336: 1358336: symbol=inflate; lookup in file=/usr/lib/x86_64-linux-gnu/perl/5.30/auto/Compress/Raw/Zlib/Zlib.so [0]
foo.1358336: 1358336: symbol=inflate; lookup in file=/lib/x86_64-linux-gnu/libz.so.1 [0]
Which yields the following:
A small rant: ELF systems (Solaris, Linux) situation is ever worse because they allow runtime symbol interposition (i.e. libraries compete for who wins over symbol names at program startup). There's no telling what symbol you will get, as the flat namespace model shows.
Some ideas to avoid any issues:
The text was updated successfully, but these errors were encountered: