Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][misc] Log Conscrypt security provider initialization warnings at debug level #23364

Merged

Conversation

lhotari
Copy link
Member

@lhotari lhotari commented Sep 28, 2024

Motivation

  • pulsar-admin output parsing breaks in 4.0.0-preview.1 testing because of the warnings

Example from Pulsar Helm chart testing

Creating subscription for output topic
2024-09-27T19:25:20,336+0000 [main] WARN  org.apache.pulsar.common.util.SecurityUtility - Conscrypt isn't available for Linux amd64. Using JDK default security provider.
Waiting for function to be ready
parse error: Invalid numeric literal at line 1, column 14

Modifications

  • switch logging level from warn to debug

Additional context

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

…t debug level

- script output parsing breaks in 4.0.0-preview.1 testing because of the warnings
@lhotari
Copy link
Member Author

lhotari commented Sep 28, 2024

This is a regression in the Alpine image since 3.3.x.

I did some manual testing to see why the native library cannot be loaded.

237d36ef299c:/tmp/META-INF/native# ldd libconscrypt_openjdk_jni-linux-x86_64.so
	/lib/ld-musl-x86_64.so.1 (0x7ffffff5d000)
	libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x7fffff06b000)
	libpthread.so.0 => /lib/ld-musl-x86_64.so.1 (0x7ffffff5d000)
	libm.so.6 => /lib/ld-musl-x86_64.so.1 (0x7ffffff5d000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7fffff047000)
	libc.so.6 => /lib/ld-musl-x86_64.so.1 (0x7ffffff5d000)
	ld-linux-x86-64.so.2 => /lib/ld-linux-x86-64.so.2 (0x7fffff011000)
Error relocating /lib/ld-linux-x86-64.so.2: unsupported relocation type 37

this is similar to #22804 error message.

and

237d36ef299c:/tmp/META-INF/native# /usr/glibc-compat/bin/ldd libconscrypt_openjdk_jni-linux-x86_64.so
./libconscrypt_openjdk_jni-linux-x86_64.so: /usr/lib/libstdc++.so.6: no version information available (required by ./libconscrypt_openjdk_jni-linux-x86_64.so)
./libconscrypt_openjdk_jni-linux-x86_64.so: /usr/lib/libstdc++.so.6: no version information available (required by ./libconscrypt_openjdk_jni-linux-x86_64.so)
./libconscrypt_openjdk_jni-linux-x86_64.so: /usr/lib/libstdc++.so.6: no version information available (required by ./libconscrypt_openjdk_jni-linux-x86_64.so)
	libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007ffffef67000)
	libpthread.so.0 => /usr/glibc-compat/lib/libpthread.so.0 (0x00007fffff7bf000)
	libm.so.6 => /usr/glibc-compat/lib/libm.so.6 (0x00007fffff6df000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fffff6bb000)
	libc.so.6 => /usr/glibc-compat/lib/libc.so.6 (0x00007ffffed25000)
	/usr/glibc-compat/lib64/ld-linux-x86-64.so.2 (0x00007ffffffca000)
	libc.musl-x86_64.so.1 => /lib/libc.musl-x86_64.so.1 (0x00007ffffec82000)

I wonder if using Alpine is the correct solution when mixing musl and glibc at runtime:
sgerrand/alpine-pkg-glibc#80 (comment)

Another comment against mixing musl and glibc at runtime:
sgerrand/alpine-pkg-glibc#194 (comment)

Yeah, fair enough - I have also seen it in cases where there is an accidental attempt to mix musl and glibc-linked binaries. I got a similar error using a musl JVM build with a different glibc native load library. Tend to agree with you that this is a common mixing problem and probably always a risk to some extent when using glibc on Alpine. You really need everything pure musl, or everything musl (but works with gcompat or libc6-compat shim-like tools...) or everything glibc using this package)

Problems like this are the reasons that a number of the folks on the Alpine team are quite against packages and approaches like this to put glibc back onto a musl-based OS such as Alpine (despite its utility they would seemingly prefer folks use a minimal glibc-based distro instead of Alpine)

The comment "You really need everything pure musl, or everything musl (but works with gcompat or libc6-compat shim-like tools...) or everything glibc using this package)" is something that is concerning. @merlimat Any thoughts on that?

@lhotari lhotari closed this Sep 28, 2024
@lhotari lhotari reopened this Sep 28, 2024
@codecov-commenter
Copy link

codecov-commenter commented Sep 28, 2024

Codecov Report

Attention: Patch coverage is 50.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 74.58%. Comparing base (bbc6224) to head (7f27e89).
Report is 609 commits behind head on master.

Files with missing lines Patch % Lines
...org/apache/pulsar/common/util/SecurityUtility.java 50.00% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #23364      +/-   ##
============================================
+ Coverage     73.57%   74.58%   +1.01%     
- Complexity    32624    33958    +1334     
============================================
  Files          1877     1934      +57     
  Lines        139502   145131    +5629     
  Branches      15299    15870     +571     
============================================
+ Hits         102638   108251    +5613     
+ Misses        28908    28581     -327     
- Partials       7956     8299     +343     
Flag Coverage Δ
inttests 27.58% <25.00%> (+2.99%) ⬆️
systests 24.58% <0.00%> (+0.26%) ⬆️
unittests 73.93% <25.00%> (+1.08%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...org/apache/pulsar/common/util/SecurityUtility.java 72.22% <50.00%> (-7.70%) ⬇️

... and 604 files with indirect coverage changes

@nodece nodece merged commit 950309b into apache:master Sep 29, 2024
106 of 108 checks passed
lhotari added a commit that referenced this pull request Oct 4, 2024
lhotari added a commit that referenced this pull request Oct 4, 2024
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Oct 15, 2024
…t debug level (apache#23364)

(cherry picked from commit 950309b)
(cherry picked from commit 988b884)
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Oct 16, 2024
…t debug level (apache#23364)

(cherry picked from commit 950309b)
(cherry picked from commit 988b884)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants