[fix][ci] Fix snappy-java native lib fails to load in x86 alpine

[yaalsn]

Motivation

From branch-3.3, pulsar starts to use alpine as base image, but it throws exception when using snappy-java lib.

Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.UnsatisfiedLinkError: /tmp/snappy-1.1.10-4b05b6fe-7374-45fc-b770-7e3728aa63de-libsnappyjava.so: Error relocating /lib/ld-linux-x86-64.so.2: unsupported relocation type 37 [in thread "pulsar-offload-write-OrderedExecutor-1-0"]

The reason is that alpine is musl-based OS while snappy-java lib built-in libsnappyjava.so is compiled with glibc.

In order to fix this issue, we need to re-compile the snappy-java native lib in alpine docker.

Modifications

Verifying this change

• Make sure that the change passes the CI checks.

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository: yaalsn#2

[coderzc]

I verified this PR and it works fine.

docker run -it \
-e PULSAR_STANDALONE_USE_ZOOKEEPER=1 \
-e PULSAR_EXTRA_OPTS="-Dzookeeper.snapshot.compression.method=snappy" \
--platform=linux/amd64 \
czcoder/pulsar:3.3.0-66baa8d \
bin/pulsar standalone

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.22%. Comparing base (bbc6224) to head (dff4565).
Report is 319 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #22804      +/-   ##
============================================
- Coverage     73.57%   73.22%   -0.35%     
+ Complexity    32624    32617       -7     
============================================
  Files          1877     1890      +13     
  Lines        139502   141644    +2142     
  Branches      15299    15541     +242     
============================================
+ Hits         102638   103721    +1083     
- Misses        28908    29917    +1009     
- Partials       7956     8006      +50     

Flag	Coverage Δ
inttests	27.37% <ø> (+2.79%)	⬆️
systests	24.75% <ø> (+0.43%)	⬆️
unittests	72.22% <ø> (-0.62%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 364 files with indirect coverage changes

[lhotari]

LGTM, left some minor comments. Good work @yaalsn

[lhotari]

Please find another solution than modifying the default bin/pulsar script

[zymap]

I don't think that this should be done for the default bin/pulsar script. There should be another way.

We don't need to add that in the OPTS by default. We can add a comment to say if you need snappy, add this -Dorg.xerial.snappy.use.systemlib=true to the PULSAR_EXTRA_OPTS

@yaalsn

[yaalsn]

We don't need to add that in the OPTS by default. We can add a comment to say if you need snappy, add this -Dorg.xerial.snappy.use.systemlib=true to the PULSAR_EXTRA_OPTS

Nice! This way is better.

[coderzc]

@lhotari @zymap PTAL

[lhotari]

We don't need to add that in the OPTS by default. We can add a comment to say if you need snappy, add this -Dorg.xerial.snappy.use.systemlib=true to the PULSAR_EXTRA_OPTS

I don't think that this is a good idea to have more configuration options to tweak. It's better to handle this in the Dockerfile building phase as I suggested by simply appending OPTS="$OPTS -Dorg.xerial.snappy.use.systemlib=true" to the /pulsar/conf/bkenv.sh file.

[coderzc]

I don't think that this is a good idea to have more configuration options to tweak. It's better to handle this in the Dockerfile building phase as I suggested by simply appending OPTS="$OPTS -Dorg.xerial.snappy.use.systemlib=true" to the /pulsar/conf/bkenv.sh file.

I also prefer this solution.

[zymap]

both ok to me.

[nodece]

One problem is that building snappy may take a lot of time.

I haven't tested it yet.

[lhotari]

One problem is that building snappy may take a lot of time.

I haven't tested it yet.

It doesn't take long to build.

[coderzc]

@lhotari PTAL