Tools are not available on cross compiled platforms since #61861

[mati865]

Moved from rust-lang/rust-clippy#4267

#61861 was merged in #61962, since then Rustfmt fails to build when cross compiling which is the case for almost all non Tier 1 platforms.
Clippy and RLS are failing with similar error.

Error on aarch64-unknown-linux-gnu:

[01:14:03] error[E0464]: multiple matching crates for `rustc_macros`
[01:14:03]   --> /cargo/registry/src/github.com-1ecc6299db9ec823/rustc-ap-rustc_target-491.0.0/abi/mod.rs:10:5
[01:14:03]    |
[01:14:03]    | use syntax_pos::symbol::{sym, Symbol};
[01:14:03]    |     ^^^^^^^^^^
[01:14:03]    |
[01:14:03]    = note: candidates:
[01:14:03]            crate `rustc_macros`: /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_macros-97bf1e9c420f9886.so
[01:14:03]            crate `rustc_macros`: /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/librustc_macros-2f4546e1163449fb.so
[01:14:03] 
[01:14:03] error[E0461]: couldn't find crate `rustc_macros` with expected target triple x86_64-unknown-linux-gnu which `syntax_pos` depends on
[01:14:03]   --> /cargo/registry/src/github.com-1ecc6299db9ec823/rustc-ap-rustc_target-491.0.0/abi/mod.rs:10:5
[01:14:03]    |
[01:14:03] 10 | use syntax_pos::symbol::{sym, Symbol};
[01:14:03]    |     ^^^^^^^^^^
[01:14:03]    |
[01:14:03]    = note: the following crate versions were found:
[01:14:03]            crate `rustc_macros`, target triple aarch64-unknown-linux-gnu: /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/aarch64-unknown-linux-gnu/release/deps/librustc_macros-55389246a2ce204b.so
[01:14:03] 
[01:14:03] error: aborting due to 2 previous errors

https://api.travis-ci.com/v3/job/209363492/log.txt

@rustbot modify labels: T-dev-tools A-rustbuild

[Manishearth]

Why does Clippy depend on rustfmt? We only have it in our Travis tests, and even then a test-fail shouldn't cause an unshipped tool

[mati865]

Modified issue description, it depends on fmt_macros but the error is basically the same.

[its-pointless]

only way i got it to build cross tools was from example in original post copying librustc_macros-97bf1e9c420f9886.so over librustc_macros-2f4546e1163449fb.so
this and suddenly rust nightly std library doubling in size... fun times.

[asomers]

ping. Any progress here? This issue blocks tier2 users from upgrading past rustc nightly 1.36.0.

[Manishearth]

I can't tell where the logs for Clippy on cross platforms are so I don't know how to fix this. Rust has a ton of builders :/

[mati865]

@Manishearth logs for last successful merge: https://dev.azure.com/rust-lang/e71b0ddf-dd27-435a-873c-e30f86eea377/_apis/build/builds/4613/logs/937?&api-version=5.0-preview

It still fails with the same error.

[Manishearth]

Hmm that seems like the rust build system being brittle. I've seen that kind of error before, it's usually solved by deleting something in the build dir.

cc @alexcrichton @pietroalbini

[alexcrichton]

That isn't an issue about stale build directories or anything like that. Something is misconfigured to cause the rustc-macros crate to get built twice. The rustc_macros crate should not be getting compiled into the stage1-tools directory, but something is pulling it in. I haven't investigated to see what, though.

(it'd be good as well btw to not assume default-flakiness and instead assume that because it is a consistent regression that there's something which needs to be investigated)

[mati865]

I think rustc_macros are expected to be available in stage1: https://github.com/rust-lang/rust/pull/58013/files#diff-4bc384d35e57dc577b9378d4d733d7c3R818
The issue started when rustfmt was updated and pulled new dependecy: rustc-ap-rustc_macros in #61861

[nico-abram]

Where should someone start looking to try to fix this, x.py?

[Lokathor]

Effectively all users were already affected as of latest stable.

[Zoxc]

-Z dual-proc-macros has the limitation that only one proc macro with the same name can be used. Making sure there's only a single version of each proc macro crate in use should fix this error.

To lift this restriction we must record the crate disambiguator of both the target and host proc macro crates when linking to it. We currently only record it for the target, so rustc can't find the right one for the host if there's multiple crates with the same name.

[gnzlbg]

This bug will be affecting many more users now that Rustup 1.20.0 installs rustfmt and clippy by default.
https://blog.rust-lang.org/2019/10/15/Rustup-1.20.0.html

For those looking for a workaround, @mati865 fixed it for libc CI which started failing due to this today by using --profile=minimal when installing rustup in rust-lang/libc#1550 .

[msizanoen1]

#65698 has been merged, now there is only rust-lang/rust-clippy#4714 and a follow-up submodule update PR (#66207) and we can close this.

[msizanoen1]

Miri, RLS and Rustfmt is now available: https://rust-lang.github.io/rustup-components-history/armv7-unknown-linux-gnueabihf.html

[Manishearth]

@msizanoen1 please don't remove the clippy plugin yet. We're waiting to publish rust-lang/blog.rust-lang.org#435 before we remove it.

[msizanoen1]

Please check the PR description carefully - it does not remove the clippy plugin but avoid building it in rustbuild.

[Manishearth]

Oh, for some reason I thought we shipped the clippy plugin package as well, but plugin users don't use rustup.

[msizanoen1]

Should be fixed by #66207

[msizanoen1]

The last tool, clippy is now available:
https://rust-lang.github.io/rustup-components-history/armv7-unknown-linux-gnueabihf.html