bootstrap rustNightly from rustBeta never works with the 'correct' channels

[qknight]

what we want

hey,
on NixOS we want to have stable, beta and nightly built as according to the docs of rust but we tried many different versions and it always fails with errors like this:

https://gist.github.com/qknight/15550e3b24b8a503897ac92fffe88804

or in short:

error[E0554]: #[feature] may not be used on the beta release channel
  --> src/libcore/lib.rs:93:1
   |
93 | #![feature(never_type)]
   | ^^^^^^^^^^^^^^^^^^^^^^^

error[E0554]: #[feature] may not be used on the beta release channel
  --> src/libcore/lib.rs:94:1
   |
94 | #![feature(prelude_import)]
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to 25 previous errors

that said have a look at these files:
https://github.com/nixcloud/nixpkgs/blob/experimentalUpdate/pkgs/development/compilers/rust/default.nix
https://github.com/nixcloud/nixpkgs/blob/experimentalUpdate/pkgs/development/compilers/rust/beta.nix
https://github.com/nixcloud/nixpkgs/blob/experimentalUpdate/pkgs/development/compilers/rust/nightly.nix

we use these channels:

• "--release-channel=stable" to build stable
• "--release-channel=beta" to build beta from stable
• "--release-channel=nightly" to build nightly from beta

and then it fails.

if we do this:

• "--release-channel=nightly" for building stable, beta and nightly we don't see the channel errors listed above.

what is going wrong here?

[the-kenny]

(I've seen this before and the following is my recollection, so it might not be completely correct)

This happens when there's a mismatch between the version used to bootstrap the compiler and what the compiler being bootstrapped expects. The current build system uses a secret to enable unstable features for stable compilers.

For a given compiler to bootstrap correctly you need to bootstrap it with the versions/snapshots specified in https://github.com/rust-lang/rust/blob/master/src/stage0.txt in the source tree of the compiler being built.

The link above specifies rustc: beta-2016-09-28 for bootstrap of today's unstable. The beta in your nix expression is way newer and isn't expected to work.

[the-kenny]

Also note that it could be related to #34722 - but that would be a rather fringe case (and I added a big fat warning to our build code to not use wrapProgram).

My bet is on the beta/stage0.txt mismatch.

[qknight]

@alexcrichton do you have an idea?

[the-kenny]

@qknight Did you try my suggestion above? I'm pretty sure it's related to the (likely) usage of incompatible versions used for bootstrapping.

On another note: @alexcrichton: We ran into this error twice now - is it possible to extend the build system of rustc to guard against this kind of misconfiguration and show a warning/error that's more clear than the can't use unstable features on stable error? If you give me a pointer where to start I'd be happy to provide a PR :)

[nagisa]

Relevant recent changes to bootstrap key logic: #37265 #37566.

If I count the releases correctly, then it should be possible to bootstrap nightly with beta just by setting RUSTC_BOOTSTRAP=1.

---

EDIT: indeed, it works

# rustup run beta rustc test.rs
error[E0554]: #[feature] may not be used on the beta release channel
 --> test.rs:1:1
  |
1 | #![feature(core_intrinsics)]
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

# set -x RUSTC_BOOTSTRAP '1'
# rustup run beta rustc test.rs

[the-kenny]

@nagisa Thanks for the pointers! :) I just enabled this proactively in our tree in NixOS/nixpkgs@bfc187f

@qknight Can you rebase/merge current master into your branch and give it a try? It should be fine now if rustc really picks up the env var :)

[qknight]

@the-kenny i will play with your advice in a bit, thanks very much!

for now i plan to include this commit into nixpkgs:

• nixcloud/nixpkgs@d695f11

why is this cool? this expression downloads a nightly rustc (no cargo) and totally bypasses hydra! this way everyone might be happy. installation in less time than a 100 mb download ;-)

[the-kenny]

+1 from my side, but other people's opinion might differ. Please note that there's also a (dev-only, not for nixpkgs) expression that fetches *todays* nightly every time it's run. Great for development :) Shall we move this discussion over to nixpkgs? Joachim Schiele <[email protected]> writes:

…

@the-kenny i will play with your advice in a bit, thanks very much! for now i plan to include this commit into nixpkgs: * nixcloud/nixpkgs@d695f11 why is this cool? this expression downloads a `nightly` rustc (no cargo) and totally bypasses hydra! this way everyone might be happy. installation in less time than a 100 mb download ;-) -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #38026 (comment)

--

[alexcrichton]

Was the bootstrap logic resolved here as well? Is this closed now?

[qknight]

thanks a lot everyone! this bug-hunt was amazing!