Check if we can strip debuginfo prior to bolting libLLVM.so #114175

lqd · 2023-07-28T12:52:48Z

r? @ghost

lqd · 2023-07-28T12:54:04Z

@bors try @rust-timer queue

bors · 2023-07-28T12:54:13Z

⌛ Trying commit 92af7b92718db72959fbe282aedbce63e5d4a657 with merge 8cfe88df5f0af2375e6e9ed4e0e370ab50282922...

bors · 2023-07-28T12:58:16Z

💔 Test failed - checks-actions

lqd · 2023-07-28T13:01:45Z

@bors try @rust-timer queue

bors · 2023-07-28T13:01:54Z

⌛ Trying commit ba180ab3f115509981bc0ead319badcb8e0e8798 with merge 11e907f9c8b9749adaf4a172268752cd929ddf0e...

bors · 2023-07-28T14:13:55Z

☀️ Try build successful - checks-actions
Build commit: 11e907f9c8b9749adaf4a172268752cd929ddf0e (11e907f9c8b9749adaf4a172268752cd929ddf0e)

rust-timer · 2023-07-29T09:44:47Z

Finished benchmarking commit (11e907f9c8b9749adaf4a172268752cd929ddf0e): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 651.8s -> 651.226s (-0.09%)

lqd · 2023-07-29T12:23:28Z

@bors try

bors · 2023-07-29T12:23:36Z

⌛ Trying commit 3f3d68db60a8364c0fa538ca85e1858c725ffff2 with merge 4a17dd658cdee54771aa956708fde5101bd0e030...

bors · 2023-07-29T13:34:33Z

☀️ Try build successful - checks-actions
Build commit: 4a17dd658cdee54771aa956708fde5101bd0e030 (4a17dd658cdee54771aa956708fde5101bd0e030)

lqd · 2023-07-30T09:30:06Z

This is not bad at all.

libLLVM-16-rust-1.73.0-nightly.so prior to #114141:

master: 173.13 MiB
this PR: 165.12 MiB (-8MiB, -4%)

libLLVM-16-rust-1.73.0-nightly.so after #114141:

master: 121.12 MiB
this PR: 113.14 MiB (still -8MiB, -6.5%)

The debuginfo in LLVM seems to be coming from libstdc++:

$ readelf --debug-dump=info .rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-16-rust-1.73.0-nightly.so | grep DW_TAG_compile_unit -A5 | grep DW_AT_comp_dir | cut -d ":" -f 2- | counts
101 counts
(  1)       39 (38.6%, 38.6%):  (indirect string, offset: 0x7): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/libsupc++
(  2)       38 (37.6%, 76.2%):  (indirect string, offset: 0x43fb2): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11
(  3)       23 (22.8%, 99.0%):  (indirect string, offset: 0x18ed8): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++98
(  4)        1 ( 1.0%,100.0%):  (indirect string, offset: 0x53f04): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src

I may make a PR of this change by itself without #114141 and see what the LLVM WG think: it doesn't seem particularly controversial to remove debuginfo coming as a side-effect when llvm itself doesn't have any. But it'd surely be preferable to not emit it in the first place if at all possible (which seems unlikely for libstdc++).

lqd · 2023-07-31T13:30:26Z

@bors try @rust-timer queue

bors · 2023-07-31T13:30:35Z

⌛ Trying commit 4d3d96a with merge 810ab570d5d27facb91806e5d9847815d9dac22a...

bors · 2023-07-31T14:43:37Z

☀️ Try build successful - checks-actions
Build commit: 810ab570d5d27facb91806e5d9847815d9dac22a (810ab570d5d27facb91806e5d9847815d9dac22a)

rust-timer · 2023-07-31T19:35:45Z

Finished benchmarking commit (810ab570d5d27facb91806e5d9847815d9dac22a): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	5.1%	[5.1%, 5.1%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.1%	[-1.2%, -1.1%]	4
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.7%	[2.7%, 2.7%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 652.068s -> 654.915s (0.44%)

lqd · 2023-07-31T19:53:05Z

Extracted into #114305

Strip unexpected debuginfo from `libLLVM.so` and `librustc_driver.so` when not requesting any debuginfo As seen in rust-lang#114175 and in [this zulip discussion](https://rust-lang.zulipchat.com/#narrow/stream/247081-t-compiler.2Fperformance/topic/Artifact.20sizes/near/379302655), there's still some small amount of debuginfo in LLVM's shared library on linux, even when not requesting it (nightly CI), coming from `libstdc++`. ``` $ readelf --debug-dump=info ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-16-rust-1.73.0-nightly.so | grep DW_TAG_compile_unit -A5 | grep DW_AT_comp_dir | cut -d ":" -f 2- | counts 101 counts ( 1) 39 (38.6%, 38.6%): (indirect string, offset: 0x7): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/libsupc++ ( 2) 38 (37.6%, 76.2%): (indirect string, offset: 0x43fb2): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11 ( 3) 23 (22.8%, 99.0%): (indirect string, offset: 0x18ed8): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++98 ( 4) 1 ( 1.0%,100.0%): (indirect string, offset: 0x53f04): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src ``` Similarly, here's `librustc_driver.so` when not requesting debuginfo from either rustc or the tools (nightly CI), coming e.g. from our LLVM wrapper: ``` $ readelf --debug-dump=info ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-e534b3a316089f5f.so | grep DW_TAG_compile_unit -A5 | grep DW_AT_comp_dir | cut -d ":" -f 2- | counts 116 counts ( 1) 34 (29.3%, 29.3%): (indirect string, offset: 0x3c11): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/libsupc++ ( 2) 32 (27.6%, 56.9%): (indirect string, offset: 0x9753c): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11 ( 3) 25 (21.6%, 78.4%): (indirect string, offset: 0x393bd): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++98 ( 4) 23 (19.8%, 98.3%): (indirect string, offset: 0x33ed3): /cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.98 ( 5) 1 ( 0.9%, 99.1%): (indirect string, offset: 0xaffff): /rustc/0d95f9132909ae7c5f2456748d0ffd1c3ba4a8e8 ( 6) 1 ( 0.9%,100.0%): (indirect string, offset: 0xb604a): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src ``` To reduce the size of distributed artifacts, this PR strips debuginfo from the LLVM and `rustc_driver` shared libraries, when: - no debuginfo is requested when building LLVM: `link-shared` is true, `optimize` is true and `release-debuginfo` is false - no debuginfo is requested when building the rustc driver: - `debuginfo-level-rustc` and `debuginfo-level-tools` are off. - when building with a stage != 0 compiler: since this is about the distributed artifacts, there's no need to do this at other stages. - for both: on a x64 linux host and target where `strip -g` is available and fixes the issue (I don't know how to strip debuginfo from a `.dylib` on mac). The LLVM BOLTed .so, and `librustc_driver.so` are big there, and this will help a little. Other targets/hosts can be added in the future if we want to. rust-lang#114175 did the same thing unconditionally in `opt-dist`, prior to BOLTing LLVM. But this should only be used in conjunction with the other config options mentioned above, and which `opt-dist` doesn't know about. Therefore, it makes more sense as in bootstrap when building LLVM and rustc when applicable and no debuginfo is requested. This shouldn't interact badly with CI caching builds and artifacts, right? --- From the other PR, `libLLVM-16-rust-1.73.0-nightly.so` prior to rust-lang#114141: - master: 173.13 MiB - stripped debuginfo: 165.12 MiB (-8 MiB, -4.6%) `libLLVM-16-rust-1.73.0-nightly.so` after rust-lang#114141: - master: 121.13 MiB - stripped debuginfo: 113.12 MiB (still -8 MiB, -6.6%) `librustc_driver.so`: - master: 118.58 MiB - stripped debuginfo: 106.46 MiB (-12 MiB, -10.2%) (Results are also available in this most recent [perf run's artifact sizes](https://perf.rust-lang.org/compare.html?start=b321edd1b2d4bd00c7b4611e8f20a03ee7b77023&end=810ab570d5d27facb91806e5d9847815d9dac22a&stat=instructions%3Au&tab=artifact-size))

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Jul 28, 2023

This comment has been minimized.

Sign in to view

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 28, 2023

lqd force-pushed the llvm-bolt-debuginfo branch from 92af7b9 to ba180ab Compare July 28, 2023 13:01

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jul 28, 2023

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jul 29, 2023

lqd force-pushed the llvm-bolt-debuginfo branch from ba180ab to 3f3d68d Compare July 29, 2023 12:21

lqd closed this Jul 30, 2023

lqd deleted the llvm-bolt-debuginfo branch July 30, 2023 09:34

lqd mentioned this pull request Jul 30, 2023

Allow stripping debuginfo from LLVM's shared library #114259

Closed

extract helper to find libLLVM's name

ae8b721

lqd added 3 commits July 31, 2023 11:32

strip debuginfo from LLVM's .so when applicable, on x64 linux

62d0845

allow DebuginfoLevel to be compared

67263d0

strip debuginfo from librustc_driver.so when applicable, on x64 linux

4d3d96a

lqd restored the llvm-bolt-debuginfo branch July 31, 2023 13:22

lqd reopened this Jul 31, 2023

lqd force-pushed the llvm-bolt-debuginfo branch from 3f3d68d to 4d3d96a Compare July 31, 2023 13:28

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jul 31, 2023

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Jul 31, 2023

lqd mentioned this pull request Jul 31, 2023

Strip unexpected debuginfo from libLLVM.so and librustc_driver.so when not requesting any debuginfo #114305

Merged

lqd closed this Jul 31, 2023

lqd deleted the llvm-bolt-debuginfo branch July 31, 2023 19:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Check if we can strip debuginfo prior to bolting libLLVM.so #114175

Check if we can strip debuginfo prior to bolting libLLVM.so #114175

lqd commented Jul 28, 2023

lqd commented Jul 28, 2023

bors commented Jul 28, 2023

This comment has been minimized.

bors commented Jul 28, 2023

lqd commented Jul 28, 2023

This comment has been minimized.

bors commented Jul 28, 2023

bors commented Jul 28, 2023

This comment has been minimized.

rust-timer commented Jul 29, 2023

lqd commented Jul 29, 2023

bors commented Jul 29, 2023

bors commented Jul 29, 2023

lqd commented Jul 30, 2023 •

edited

Loading

lqd commented Jul 31, 2023

This comment has been minimized.

bors commented Jul 31, 2023

bors commented Jul 31, 2023

This comment has been minimized.

rust-timer commented Jul 31, 2023

lqd commented Jul 31, 2023

Check if we can strip debuginfo prior to bolting libLLVM.so #114175

Check if we can strip debuginfo prior to bolting libLLVM.so #114175

Conversation

lqd commented Jul 28, 2023

lqd commented Jul 28, 2023

bors commented Jul 28, 2023

This comment has been minimized.

bors commented Jul 28, 2023

lqd commented Jul 28, 2023

This comment has been minimized.

bors commented Jul 28, 2023

bors commented Jul 28, 2023

This comment has been minimized.

rust-timer commented Jul 29, 2023

Overall result: no relevant changes - no action needed

lqd commented Jul 29, 2023

bors commented Jul 29, 2023

bors commented Jul 29, 2023

lqd commented Jul 30, 2023 • edited Loading

lqd commented Jul 31, 2023

This comment has been minimized.

bors commented Jul 31, 2023

bors commented Jul 31, 2023

This comment has been minimized.

rust-timer commented Jul 31, 2023

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

lqd commented Jul 31, 2023

lqd commented Jul 30, 2023 •

edited

Loading