Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split out LLVM PGO step and use clang 13 to compile LLVM #89499

Merged
merged 2 commits into from
Oct 18, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/ci/docker/host-x86_64/dist-x86_64-linux/build-clang.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ set -ex

source shared.sh

LLVM=llvmorg-12.0.1
LLVM=llvmorg-13.0.0

mkdir llvm-project
cd llvm-project
Expand Down
38 changes: 30 additions & 8 deletions src/ci/pgo.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,13 @@ set -euxo pipefail

rm -rf /tmp/rustc-pgo

# We collect LLVM profiling information and rustc profiling information in
# separate phases. This increases build time -- though not by a huge amount --
# but prevents any problems from arising due to different profiling runtimes
# being simultaneously linked in.

python3 ../x.py build --target=$PGO_HOST --host=$PGO_HOST \
--stage 2 library/std \
--rust-profile-generate=/tmp/rustc-pgo \
--llvm-profile-generate

# Profile libcore compilation in opt-level=0 and opt-level=3
Expand All @@ -15,6 +19,29 @@ RUSTC_BOOTSTRAP=1 ./build/$PGO_HOST/stage2/bin/rustc --edition=2018 \
RUSTC_BOOTSTRAP=1 ./build/$PGO_HOST/stage2/bin/rustc --edition=2018 \
--crate-type=lib -Copt-level=3 ../library/core/src/lib.rs

# Merge the profile data we gathered for LLVM
# Note that this uses the profdata from the clang we used to build LLVM,
# which likely has a different version than our in-tree clang.
/rustroot/bin/llvm-profdata \
merge -o /tmp/llvm-pgo.profdata ./build/$PGO_HOST/llvm/build/profiles

# Rustbuild currently doesn't support rebuilding LLVM when PGO options
# change (or any other llvm-related options); so just clear out the relevant
# directories ourselves.
rm -r ./build/$PGO_HOST/llvm ./build/$PGO_HOST/lld

# Okay, LLVM profiling is done, switch to rustc PGO.

python3 ../x.py build --target=$PGO_HOST --host=$PGO_HOST \
--stage 2 library/std \
--rust-profile-generate=/tmp/rustc-pgo

# Profile libcore compilation in opt-level=0 and opt-level=3
RUSTC_BOOTSTRAP=1 ./build/$PGO_HOST/stage2/bin/rustc --edition=2018 \
--crate-type=lib ../library/core/src/lib.rs
RUSTC_BOOTSTRAP=1 ./build/$PGO_HOST/stage2/bin/rustc --edition=2018 \
--crate-type=lib -Copt-level=3 ../library/core/src/lib.rs

cp -r /tmp/rustc-perf ./
chown -R $(whoami): ./rustc-perf
cd rustc-perf
Expand Down Expand Up @@ -46,18 +73,13 @@ cd /checkout/obj
./build/$PGO_HOST/llvm/bin/llvm-profdata \
merge -o /tmp/rustc-pgo.profdata /tmp/rustc-pgo

# Merge the profile data we gathered for LLVM
# Note that this uses the profdata from the clang we used to build LLVM,
# which likely has a different version than our in-tree clang.
/rustroot/bin/llvm-profdata \
merge -o /tmp/llvm-pgo.profdata ./build/$PGO_HOST/llvm/build/profiles

# Rustbuild currently doesn't support rebuilding LLVM when PGO options
# change (or any other llvm-related options); so just clear out the relevant
# directories ourselves.
rm -r ./build/$PGO_HOST/llvm ./build/$PGO_HOST/lld

# This produces the actual final set of artifacts.
# This produces the actual final set of artifacts, using both the LLVM and rustc
# collected profiling data.
$@ \
--rust-profile-use=/tmp/rustc-pgo.profdata \
--llvm-profile-use=/tmp/llvm-pgo.profdata