Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow stripping debuginfo from LLVM's shared library #114259

Closed
wants to merge 6 commits into from

Conversation

lqd
Copy link
Member

@lqd lqd commented Jul 30, 2023

As seen in #114175, there's still some small amount of debuginfo in LLVM's shared library on linux, even when not requesting it (much like in librustc_driver.so), coming from libstdc++.

$ readelf --debug-dump=info .rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-16-rust-1.73.0-nightly.so | grep DW_TAG_compile_unit -A5 | grep DW_AT_comp_dir | cut -d ":" -f 2- | counts
101 counts
(  1)       39 (38.6%, 38.6%):  (indirect string, offset: 0x7): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/libsupc++
(  2)       38 (37.6%, 76.2%):  (indirect string, offset: 0x43fb2): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11
(  3)       23 (22.8%, 99.0%):  (indirect string, offset: 0x18ed8): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++98
(  4)        1 ( 1.0%,100.0%):  (indirect string, offset: 0x53f04): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src

This PR adds a llvm.strip-debuginfo config to strip it away, when all of the following hold:

  • this config option is explicitly set to true, the default is false
  • llvm.link-shared is true
  • llvm.release-debuginfo is false
  • on a linux target (I don't know how to strip debuginfo from a dylib on mac): the BOLTed .so is huge, so this will help a little.

#114175 did the same thing unconditionally in opt-dist, prior to BOLTing LLVM. But this should be validated and used in conjunction with these other config options above, and which opt-dist doesn't know about. Therefore, it makes more sense as a dedicated option, to be used in bootstrap when building LLVM (and enabled in CI on e.g. the dist-x86_64-linux builder).


From the other PR, libLLVM-16-rust-1.73.0-nightly.so prior to #114141:

  • master: 173.13 MiB
  • stripped debuginfo: 165.12 MiB (-8MiB, -4%)

And from this PR, libLLVM-16-rust-1.73.0-nightly.so after #114141:

  • master: 121.14 MiB
  • stripped debuginfo: 113.14 MiB (still -8MiB, -6.5%)

(Draft while I do some try builds to ensure this is working as expected on CI, and until I ask e.g. cuviper whether it's a good idea 😓. Or if we could avoid emitting that debuginfo in the first place, if at all possible).

(After we validate whether it's interesting, I could add similar mechanism for librustc_driver.so, which also has some debuginfo from libstdc++, as well as compiler_builtins: I tested locally that removing that would reduce its size by about 10%.)

(I also don't know if it should be added to src/bootstrap/configure.py, I don't think so. And may also need its check_ci_llvm! to be removed, where CI needs download-ci-llvm...)

@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Jul 30, 2023
@lqd
Copy link
Member Author

lqd commented Jul 30, 2023

@bors try

@bors
Copy link
Contributor

bors commented Jul 30, 2023

⌛ Trying commit 2ba4bae with merge a9b44723d564723ba744f3dce36c875d5c4786c8...

@bors
Copy link
Contributor

bors commented Jul 30, 2023

☀️ Try build successful - checks-actions
Build commit: a9b44723d564723ba744f3dce36c875d5c4786c8 (a9b44723d564723ba744f3dce36c875d5c4786c8)

@Kobzol
Copy link
Contributor

Kobzol commented Jul 31, 2023

This flag is a bit hacky, because it basically just removes pre-existing debuginfo that isn't generated by bootstrap. And if we're going to only do it for Linux and only for LLVM built as a shared library, which is a pretty specific use-case, it would make more sense to me to have it in opt-dist.

Alternatively, maybe we could just always strip libLLVM.so if debuginfo is not requested? It's weird to have a default which produces some quasi-debuginfo, a flag (debuginfo) that enables full debuginfo, and then another niche flag that generates the state that should be the default (no debuginfo at all). We could also strip the C++ stdlib library before even building anything with bootstrap, since the debuginfo symbols will be currently included into librustc_driver.so, I think.

@lqd
Copy link
Member Author

lqd commented Jul 31, 2023

Ok I'll do this without a flag.

@lqd lqd closed this Jul 31, 2023
@lqd lqd deleted the llvm-debuginfo branch July 31, 2023 10:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants