add CROSS_CUSTOM_TOOLCHAIN to disable automatic target component downloading

[Emilgardis]

This adds an environment variable CROSS_CUSTOM_TOOLCHAIN to disable some rustup commands.

The way to use this with cargo-bisect-rustc is cargo bisect-rustc --script=./bisect.sh --target powerpc64le-unknown-linux-gnu and the script is

#!/usr/bin/env bash

export CROSS_CUSTOM_TOOLCHAIN=1
exec cross run --target powerpc64le-unknown-linux-gnu

I've filed rust-lang/cargo-bisect-rustc#159 to avoid the pitfall that was discovered when not specifying --target

resolves #699

[Emilgardis]

ping @NobodyXu

does this work for you? I've been trying to replicate your errors but I can't manage to do it. That to me means its either an indication of me not understanding the issue, something being fishy with our implementation of PPCle or something sometimes working and sometimes not.

Anyway, this should solve the issue of using cross with a custom toolchain (that is rustup compatible)

[Alexhuszagh]

I don't think this will work because the custom toolchains are installed dynamically and won't necessarily have the desired target added for cross-compiling. This would work if the custom toolchain was installed by the user with every feature desired already, but cargo-bisect-rustc doesn't do that I believe.

For cargo-bisect-rustc, the commit IDs are all valid, so I think we can work on that, and have a fallthrough for other cases for custom toolchains. In short, it's a "custom toolchain" but we can still manually install the components for it, if we so desire.

[Emilgardis]

cargo-bisect-rustc plays nice here and installs everything that we explicitly install ourselves in the commands "protected" by the env var (except clippy)

I think I've tried it enough, it works

[Emilgardis]

The way to use this with cargo-bisect-rustc is cargo bisect-rustc --script=./bisect.sh --target powerpc64le-unknown-linux-gnu and the script is

#!/usr/bin/env bash

export CROSS_CUSTOM_TOOLCHAIN=1
exec cross run --target powerpc64le-unknown-linux-gnu

[NobodyXu]

Let me have a try.

[Alexhuszagh]

Once I finish a few commits, I'll try as well with a guaranteed failing command and see how it works.

[NobodyXu]

Worked on my machine:

$ cargo bisect-rustc -vv --script=./test.sh --target powerpc64le-unknown-linux-gnu
installing nightly-2022-06-16
testing...
+ export CROSS_CUSTOM_TOOLCHAIN=1
+ CROSS_CUSTOM_TOOLCHAIN=1
+ exec cross run -vv --target powerpc64le-unknown-linux-gnu
+ "/home/nobodyxu/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo" "metadata" "--format-version=1" "--filter-platform" "powerpc64le-unknown-linux-gnu"
+ "rustc" "--print" "sysroot"
+ "rustup" "toolchain" "list"
+ "/usr/bin/docker" "--help"
+ "/usr/bin/docker" "run" "--userns" "host" "-e" "PKG_CONFIG_ALLOW_CROSS=1" "-e" "XARGO_HOME=/xargo" "-e" "CARGO_HOME=/cargo" "-e" "CARGO_TARGET_DIR=/target" "-e" "CROSS_RUNNER=" "-e" "USER=nobodyxu" "--rm" "--user" "1000:1000" "-v" "/home/nobodyxu/.xargo:/xargo:Z" "-v" "/home/nobodyxu/.cargo:/cargo:Z" "-v" "/cargo/bin" "-v" "/home/nobodyxu/tests:/project:Z" "-v" "/home/nobodyxu/.rustup/toolchains/bisector-nightly-2022-06-16-x86_64-unknown-linux-gnu:/rust:Z,ro" "-v" "/home/nobodyxu/tests/target-bisector-nightly-2022-06-16-x86_64-unknown-linux-gnu:/target:Z" "-w" "/project" "ghcr.io/cross-rs/powerpc64le-unknown-linux-gnu:main" "sh" "-c" "PATH=$PATH:/rust/bin cargo run -vv --target powerpc64le-unknown-linux-gnu"
   Compiling tests v0.1.0 (/project)
     Running `CARGO=/rust/bin/cargo CARGO_BIN_NAME=tests CARGO_CRATE_NAME=tests CARGO_MANIFEST_DIR=/project CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=tests CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH='/target/debug/deps:/rust/lib' rustc --crate-name tests --edition=2018 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=1eaf9da53e7e178c -C extra-filename=-1eaf9da53e7e178c --out-dir /target/powerpc64le-unknown-linux-gnu/debug/deps --target powerpc64le-unknown-linux-gnu -C linker=powerpc64le-linux-gnu-gcc -C incremental=/target/powerpc64le-unknown-linux-gnu/debug/incremental -L dependency=/target/powerpc64le-unknown-linux-gnu/debug/deps -L dependency=/target/debug/deps`
    Finished dev [unoptimized + debuginfo] target(s) in 0.55s
     Running `/linux-runner powerpc64le /target/powerpc64le-unknown-linux-gnu/debug/tests`
3.2
-inf
RESULT: nightly-2022-06-16, ===> No
uninstalling nightly-2022-06-16

and my test.sh:

#!/bin/bash

set -euxo pipefail

export CROSS_CUSTOM_TOOLCHAIN=1
exec cross run -vv --target powerpc64le-unknown-linux-gnu

[Emilgardis]

Worked on my machine:

Awesome, however the interesting part is when it installs an "non-normal" toolchain, e.g one that rustup reports as a custom toolchain (i.e installed with rustup toolchain link)

In an unrelated note to this specific pr, I can't replicate the behaviour you suggest in your rustc issue, e.g I can't find a date which reproduces the issue of 3.2_f32 printing as 0

[Alexhuszagh]

Nevermind this does work as long as you provide the --target both on the outside and inside. lgtm. I'll add this to the things-to-add-to-the-wiki as well.

[Emilgardis]

Nevermind this does work as long as you provide the --target both on the outside and inside. lgtm. I'll add this to the things-to-add-to-the-wiki as well.

don't think you need to add it, I've filed rust-lang/cargo-bisect-rustc#159 which would fix it automagically (with #804)

[Alexhuszagh]

Worked on my machine:

Awesome, however the interesting part is when it installs an "non-normal" toolchain, e.g one that rustup reports as a custom toolchain (e.g installed with rustup toolchain link)

In an unrelated note to this specific pr, I can't replicate the behaviour you suggest in your rustc issue, e.g I can't find a date which reproduces the issue of 3.2_f32 printing as 0

This was a Qemu error 🙃. This was actually the exact bug for the exact target I started contributing to this repository in the first place.

[NobodyXu]

Worked on my machine:

Awesome, however the interesting part is when it installs an "non-normal" toolchain, e.g one that rustup reports as a custom toolchain (e.g installed with rustup toolchain link)
In an unrelated note to this specific pr, I can't replicate the behaviour you suggest in your rustc issue, e.g I can't find a date which reproduces the issue of 3.2_f32 printing as 0

This was a Qemu error 🙃. This was actually the exact bug for the exact target I started contributing to this repository in the first place.

Oops

Never thought it was a Qemu error...
Is that because the Qemu used by cross is too old?

Edit:

If so, then why does it succeeded after I upgraded the nightly toolchain but yet failed again after some upgrade?
Also, I wasn't able to reproduce it on stable...

[Alexhuszagh]

lgtm. I'm assuming we want to just check if the envvar is set, rather than it's truthy?

[Emilgardis]

the specific qemu error was fixed as part of #609

there might have been a mix of qemu errors and alignment errors on rustc, hard to say for sure without knowing exactly what was done

[Alexhuszagh]

The old releases for cross used a much older Qemu version. We (as in I, I'll accept that mistake) then pinned it to Qemu 3.x because of a similar bug, but this causes other issues that were fixed in 4.x. We now installed off-of-main use 5.x, which fixes all these issues.

[Emilgardis]

lgtm. I'm assuming we want to just check if the envvar is set, rather than it's truthy?

I think (hope) no one sets it when not needed, worst case that happens when it's set is that the user doesn't get easy downloading of needed components

lets ship it

bors r=Alexhuszagh

[NobodyXu]

It totally didn't occur to me that it involves Qemu bugs

[Emilgardis]

It totally didn't occur to me that it involves Qemu bugs

;) do note also that cross, while using newer qemu, doesn't still use the newest, for that we need #587

[bors]

Build succeeded:

• conclusion