crc32/adler32_combine doesn't work on 32-bit musl systems

[rrbq]

Hello, while building Perl 5.36 on Alpine Linux, I noticed that the testsuite for C:R:Z fails the crc32/adler32_combine tests on 32-bit platforms (specifically x86, armv7, and armhf).

Alpine uses musl libc instead of glibc. When building Perl, Alpine's build system sets the environment variable BUILD_ZLIB=0, and removes C:R:Z's zlib-src directory.

From what I could understand of this issue, it has something to do with musl defining off_t to be 64-bit everywhere, even on 32-bit platforms.

This breaks the assumption that off_t always has the same size as long, which while true for 64-bit systems, is no longer the case for 32-bit systems using musl.

On 32-bit Alpine, this results in its system zlib having a 64-bit z_off_t, while C:R:Z expects a 32-bit z_off_t. For those platforms, the crc32/adler32_combine functions are also not functioning properly with the C:R:Z bundled in previous versions of Perl, but as the tests for those functions just appeared in Perl 5.36, Alpine has not noticed this problem until now.

After some discussion, the Alpine developers have decided to provisionally remove -DZ_SOLO from the Makefile.PL of C:R:Z. To reach this decision, it was observed that there are 2 places in zlib's zconf.h where z_off_t could get #defined:

#ifndef Z_SOLO
#  if defined(Z_HAVE_UNISTD_H) || defined(_LARGEFILE64_SOURCE)
#    include <unistd.h>         /* for SEEK_*, off_t, and _LFS64_LARGEFILE */
#    ifdef VMS
#      include <unixio.h>       /* for off_t */
#    endif
#    ifndef z_off_t
#      define z_off_t off_t
#    endif
#  endif
#endif

and

#ifndef z_off_t
#  define z_off_t long
#endif

When Z_SOLO is defined, the first definition of z_off_t (which would give the right size of 64-bit everywhere for systems using musl) is ignored, and instead the second one is used, which causes problems on 32-bit systems.

I have some questions about this:

1. May I know if it is safe to remove -DZ_SOLO while building C:R:Z? Are there any unexpected behaviors we should be on the look out for?
2. If it is safe to do so, would it be possible for C:R:Z to stop defining Z_SOLO whenever a pre-built system zlib is used (just like what happens with Z_PREFIX)?

Thank you.

[pmqs]

Hey @rrbq, thanks for the bug report.

I have some questions about this:

1. May I know if it is safe to remove -DZ_SOLO while building C:R:Z? Are there any unexpected behaviors we should be on the look out for?

if you aren't building C:R:Z with the zlib source included, than it will be fine.

2. If it is safe to do so, would it be possible for C:R:Z to stop defining Z_SOLO whenever a pre-built system zlib is used (just like what happens with Z_PREFIX)?

Yes, that looks like the best way forward with this issue. The use of Z_SOLO only makes sense when C:R:Z is building its own copy if zlib.

I'll get an update to my code out

[pmqs]

@rrbq if you get a chance could you try the latest version of the code I've added to github please? If that works for you I'll spin an official release.

[rrbq]

Thanks for the bug fix. I can confirm that the latest c44e0b7 on Github fixes the problem I had with Z_SOLO on 32-bit musl systems.

[pmqs]

Change 5338226 adds a 32-bit Alpine workflow file, alpine-32bit.yml, that will guard against regressions with this issue.

[pmqs]

Thanks for the bug fix. I can confirm that the latest c44e0b7 on Github fixes the problem I had with Z_SOLO on 32-bit musl systems.

Thanks @rrbq. Just getting some infrastructure in place to test for regressions with this fix in place before I push out an official release.

[pmqs]

Issue fixed in Compress::Raw::Zlib 2.200. Uploaded to CPAN (https://metacpan.org/dist/Compress-Raw-Zlib)