improve compiler&tool documenting and re-enable cranelift on CI

[onur-ozkan]

First commit addresses the linking issue with compiler crates. Second one ensures that compiler crates are linked correctly (with later commits we added this check for tools as well), allowing us to detect these hard-to-catch bugs on CI. Following three commits cherry-picked from #117328 to re-enable the Cranelift backend on CI.

More info: #117430

cc @bjorn3 @RalfJung

[rustbot]

r? @Mark-Simulacrum

(rustbot has picked a reviewer for you, use r? to override)

[bjorn3]

Thanks a lot for working on this!

Could you add the following patch? I found this unintended change while I was (unsuccessfully) trying to figure out the doc issue too.

Author: bjorn3 <[email protected]>
Date:   Sat Nov 4 12:47:05 2023 +0000

    Don't disable inline asm usage in compiler-builtins when the cranelift backend is enabled
    
    This was a leftover from when inline asm wasn't universally supported by
    the cranelift backend yet. It would cause the optimized inline asm
    intrinsics to be avoided for the standard library distributed with
    rustup now that we build the cranelift backend by default on nightly for
    several tier 1 targets.

diff --git a/src/bootstrap/src/core/build_steps/compile.rs b/src/bootstrap/src/core/build_steps/compile.rs
index 1eed534150b..8ab0ef61c50 100644
--- a/src/bootstrap/src/core/build_steps/compile.rs
+++ b/src/bootstrap/src/core/build_steps/compile.rs
@@ -413,11 +413,6 @@ pub fn std_cargo(builder: &Builder<'_>, target: TargetSelection, stage: u32, car
 
     let mut features = String::new();
 
-    // Cranelift doesn't support `asm`.
-    if stage != 0 && builder.config.default_codegen_backend().unwrap_or_default() == "cranelift" {
-        features += " compiler-builtins-no-asm";
-    }
-
     if builder.no_std(target) == Some(true) {
         features += " compiler-builtins-mem";
         if !target.starts_with("bpf") {

[onur-ozkan]

Thanks a lot for working on this!

Could you add the following patch? I found this unintended change while I was (unsuccessfully) trying to figure out the doc issue too.

Author: bjorn3 <[email protected]>
Date:   Sat Nov 4 12:47:05 2023 +0000

    Don't disable inline asm usage in compiler-builtins when the cranelift backend is enabled
    
    This was a leftover from when inline asm wasn't universally supported by
    the cranelift backend yet. It would cause the optimized inline asm
    intrinsics to be avoided for the standard library distributed with
    rustup now that we build the cranelift backend by default on nightly for
    several tier 1 targets.

diff --git a/src/bootstrap/src/core/build_steps/compile.rs b/src/bootstrap/src/core/build_steps/compile.rs
index 1eed534150b..8ab0ef61c50 100644
--- a/src/bootstrap/src/core/build_steps/compile.rs
+++ b/src/bootstrap/src/core/build_steps/compile.rs
@@ -413,11 +413,6 @@ pub fn std_cargo(builder: &Builder<'_>, target: TargetSelection, stage: u32, car
 
     let mut features = String::new();
 
-    // Cranelift doesn't support `asm`.
-    if stage != 0 && builder.config.default_codegen_backend().unwrap_or_default() == "cranelift" {
-        features += " compiler-builtins-no-asm";
-    }
-
     if builder.no_std(target) == Some(true) {
         features += " compiler-builtins-mem";
         if !target.starts_with("bpf") {

It gives "fatal: empty ident name (for <>) not allowed" when I try to git am on the patch. Is it ok for you if I do git apply on it (it will show my name instead of yours on the commit)?

[bjorn3]

You could try git commit --amend --author "bjorn3 <[email protected]>" to change the author of the commit. If that doesn't work, plain git apply is fine with me.

[onur-ozkan]

Seems like rustdoc and rustdoc_json_types docs are also missing. Let's block the PR until I fix them.

@rustbot blocked

[Mark-Simulacrum]

In general r=me, but happy to see this build docs in CI (via a try build, likely with a change like this to re-enable docs: 43f592f) before we land it. Happy to re-review with the extra fix you mention (but not strictly needed if similar changes).

[onur-ozkan]

@bors try

[bors]

⌛ Trying commit a348b00 with merge 4d1b721...

[bors]

💔 Test failed - checks-actions

[onur-ozkan]

@bors try

[bors]

⌛ Trying commit a3fe394 with merge 83e9fc5...

[bors]

💔 Test failed - checks-actions

[onur-ozkan]

@bors try

[bors]

⌛ Trying commit 32946d2 with merge dc8c831...

[bors]

☀️ Try build successful - checks-actions
Build commit: dc8c831 (dc8c8311ce31a9e8e92f63c098be23b53f2bfc21)

[onur-ozkan]

Here the try build that includes compiler docs: https://ci-artifacts.rust-lang.org/rustc-builds/dc8c8311ce31a9e8e92f63c098be23b53f2bfc21/rustc-docs-nightly-x86_64-unknown-linux-gnu.tar.xz. However, it's missing rustdoc pages. I will trigger bors for another try build (I can't debug this build locally, working with a laptop atm) to see if this issue gets resolved. If not, I will continue working on it tonight or early tomorrow on my desktop (so I can debug things faster).

EDIT: It seems the last attempt fixed rustdoc problem!

[onur-ozkan]

@bors try

[bors]

⌛ Trying commit 43bc2dd with merge d4862cd...

[bors]

☀️ Try build successful - checks-actions
Build commit: d4862cd (d4862cdec9eae562e5318823f9a7374b10d6acb6)

[onur-ozkan]

It's ready now! Here is the build https://ci-artifacts.rust-lang.org/rustc-builds/d4862cdec9eae562e5318823f9a7374b10d6acb6/rustc-docs-nightly-x86_64-unknown-linux-gnu.tar.xz

[onur-ozkan]

Happy to re-review with the extra fix you mention (but not strictly needed if similar changes).

No strict changes have been applied. Also, we are able to catch missing-doc problem for compiler and tool crates (like here https://github.com/rust-lang-ci/rust/actions/runs/6760207883/job/18373700086#step:25:28933), so it should be fine to r.

@bors r=Mark-Simulacrum rollup=never

[bors]

📌 Commit b0df821 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

[bors]

⌛ Testing commit b0df821 with merge 6c7de31...

[bors]

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing 6c7de31 to master...

[rust-timer]

Finished benchmarking commit (6c7de31): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.7%	[1.6%, 3.7%]	5
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.1%	[-1.1%, -1.0%]	2
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.6%	[-1.1%, 3.7%]	7

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.6%	[0.5%, 0.6%]	2
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.5%	[-0.6%, -0.5%]	2
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.0%	[-0.6%, 0.6%]	4

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 635.747s -> 638.861s (0.49%)
Artifact size: 304.50 MiB -> 304.51 MiB (0.00%)