Work around dlopen not working properly under sanitizer instrumentation #46255
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Apple has an API for exactly this purpose, but it is private: https://github.com/apple-oss-distributions/dyld/blob/5c9192436bb195e7a8fe61f22a229ee3d30d8222/include/dlfcn_private.h#L53-L58 |
`dlopen` has a mis-feature where it looks at the return address to determine the calling object to look at it's RUNPATH. Because asan intercepts `dlopen`, the calling object check finds asan rather than julia, causing an incorrect RUNPATH (and other flags to be used). Arguably, this is mostly a libc problem, because there isn't really a way to directly specify the resolution scope. I have sent a proposal to libc-coord [1] to fix this, but of course, we can't wait for that to percolate down to us. Instead, this takes advantage of the fact that almost all of our dlopen calls go through a single entrypoint in jl_dlopen, so we can insert additional logic here to make this work. This doesn't catch uses of `dlopen` in jlls (which is a problem for things like plugin loading in various jlls), but it at least makes base julia work. We can punt the jll question to another day - either with a patched libc in PkgEval or by patching the jll source with an analogous patch. Regardless, with this, Julia bootstraps properly under asan, without any special LD_LIBRARY_PATH hacks. [1] https://www.openwall.com/lists/libc-coord/2022/08/04/1
pcjentsch
pushed a commit
to pcjentsch/julia
that referenced
this pull request
Aug 18, 2022
…on (JuliaLang#46255) `dlopen` has a mis-feature where it looks at the return address to determine the calling object to look at it's RUNPATH. Because asan intercepts `dlopen`, the calling object check finds asan rather than julia, causing an incorrect RUNPATH (and other flags to be used). Arguably, this is mostly a libc problem, because there isn't really a way to directly specify the resolution scope. I have sent a proposal to libc-coord [1] to fix this, but of course, we can't wait for that to percolate down to us. Instead, this takes advantage of the fact that almost all of our dlopen calls go through a single entrypoint in jl_dlopen, so we can insert additional logic here to make this work. This doesn't catch uses of `dlopen` in jlls (which is a problem for things like plugin loading in various jlls), but it at least makes base julia work. We can punt the jll question to another day - either with a patched libc in PkgEval or by patching the jll source with an analogous patch. Regardless, with this, Julia bootstraps properly under asan, without any special LD_LIBRARY_PATH hacks. [1] https://www.openwall.com/lists/libc-coord/2022/08/04/1
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
dlopenhas a mis-feature where it looks at the return address to determinethe calling object to look at it's RUNPATH. Because asan intercepts
dlopen,the calling object check finds asan rather than julia, causing an incorrect
RUNPATH (and other flags to be used). Arguably, this is mostly a libc problem,
because there isn't really a way to directly specify the resolution scope.
I have sent a proposal to libc-coord [1] to fix this, but of course, we can't
wait for that to percolate down to us. Instead, this takes advantage of the
fact that almost all of our dlopen calls go through a single entrypoint in
jl_dlopen, so we can insert additional logic here to make this work. This
doesn't catch uses of
dlopenin jlls (which is a problem for things likeplugin loading in various jlls), but it at least makes base julia work.
We can punt the jll question to another day - either with a patched libc
in PkgEval or by patching the jll source with an analogous patch.
Regardless, with this, Julia bootstraps properly under asan, without any
special LD_LIBRARY_PATH hacks.
[1] https://www.openwall.com/lists/libc-coord/2022/08/04/1