Incremental compilation error building rustc (-i --keep-stage 0)
#59105
Labels
A-incr-comp
Area: Incremental compilation
C-bug
Category: This is a bug.
P-medium
Medium priority
T-bootstrap
Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
T-compiler
Relevant to the compiler team, which will review and decide on the PR/issue.
Comments
jonas-schievink
added
A-incr-comp
|
Stopgap: |
|
@estebank Yeah I know, but then I have to compile from scratch, and it takes forever. :-( |
|
I think I see one way this happens. I think the issue is that Here is one way to repro this:
The last step should fail in stage1 to build I think the solution is to clear the host directory, although care should be taken so it is not cleared multiple times if there are multiple targets. I'm not super familiar with rustbuild, but maybe @Mark-Simulacrum or @collin5 know more about this. One way to get this particular example back on track is to Some similar issues: #49979, #50481 EDIT: Other things like codegen have to be cleaned out too, like |
|
@estebank Do you know who works on incremental compilation in the compiler team? This seems like a pretty high-priority bug, so maybe we should tag them. :-) |
|
I believe @alexcrichton, @nikomatsakis, @Zoxc, @michaelwoerister and @MarkSimulacrum have been involved with incremental compilation at one point or another. |
|
Based on the error message, this likely has nothing to do with the compiler -- it's most likely purely a problem inside rustbuild (i.e., rustbuild is broken). |
|
Yea, this is only related to rustbuild. The issue is that cargo does not track libstd as a dependency, so rustbuild needs to be careful to force rebuilding of crates when libstd changes. It does that currently (via |
|
Ah okay. So it should be quite a straightforward fix for someone who knows how rustbuild works, eh? |
|
Actually quite difficult, but yes, in theory it's a fairly easy fix. |
pnkfelix
changed the title
-i --keep-stage 0)
Centril
added
I-nominated
T-compiler
|
triage: leaving I-nominated for discussion of prioritization at T-compiler meeting itself. (My personal inclination is to assign this P-medium priority, and if no one takes it on at the meeting or shortly thereafter, I may self-assign.) |
|
discussed at T-compiler meeting. Assigning P-medium priority, since this is essentially an internal tooling issue that we would like to fix, but does not take precedence over customer P-high matters. |
|
@pnkfelix Sounds reasonable enough. It's probably P-high by rustc developer standards, but when you consider the scale has to include end-users too, P-medium makes more sense. |
|
Currently I'm not able to build the Rust project. Even running I received this error: Checking test artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Checking rustc_term v0.0.1
Checking getopts v0.2.17
Checking proc_macro v0.0.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/libproc_macro)
Checking libtest v0.0.1
error[E0460]: found possibly newer version of crate `std` which `getopts` depends onest
--> /home/lzutao/.cargo/registry/src/github.com-1ecc6299db9ec823/libtest-0.0.1/lib.rs:34:5
|
34 | use getopts;
| ^^^^^^^
|
= note: perhaps that crate needs to be recompiled?
= note: the following crate versions were found:
crate `std`: /home/lzutao/github.com/lzutao/rust-fork/rust/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d26c9396879f81c2.rmeta
crate `getopts`: /home/lzutao/github.com/lzutao/rust-fork/rust/build/x86_64-unknown-linux-gnu/stage0-test/x86_64-unknown-linux-gnu/release/deps/libgetopts-267ef145c76bc717.rmeta
error: aborting due to previous errorEDIT: It might be related that I'm setting Full stdout when running `./x.py check`% ./x.py check
Updating only changed submodules
Submodules updated in 0.04 seconds
downloading https://static.rust-lang.org/dist/2019-03-20/rust-std-beta-x86_64-unknown-linux-gnu.tar.gz
##################################################################################################### 100.0%
extracting /home/lzutao/github.com/lzutao/rust-fork/rust/build/cache/2019-03-20/rust-std-beta-x86_64-unknown-linux-gnu.tar.gz
downloading https://static.rust-lang.org/dist/2019-03-20/rustc-beta-x86_64-unknown-linux-gnu.tar.gz
##################################################################################################### 100.0%
extracting /home/lzutao/github.com/lzutao/rust-fork/rust/build/cache/2019-03-20/rustc-beta-x86_64-unknown-linux-gnu.tar.gz
downloading https://static.rust-lang.org/dist/2019-03-20/cargo-beta-x86_64-unknown-linux-gnu.tar.gz
##################################################################################################### 100.0%
extracting /home/lzutao/github.com/lzutao/rust-fork/rust/build/cache/2019-03-20/cargo-beta-x86_64-unknown-linux-gnu.tar.gz
Compiling proc-macro2 v0.4.24
Compiling unicode-xid v0.1.0
Compiling ryu v0.2.7
Compiling libc v0.2.51
Compiling serde v1.0.82
Compiling itoa v0.4.3
Compiling fixedbitset v0.1.9
Compiling cc v1.0.28
Compiling cfg-if v0.1.6
Compiling ordermap v0.3.5
Compiling build_helper v0.1.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/build_helper)
Compiling lazy_static v0.2.11
Compiling getopts v0.2.17
Compiling cmake v0.1.33
Compiling petgraph v0.4.13
Compiling quote v0.6.10
Compiling num_cpus v1.8.0
Compiling filetime v0.2.4
Compiling time v0.1.40
Compiling serde_json v1.0.33
Compiling toml v0.4.10
Compiling syn v0.15.22
Compiling serde_derive v1.0.81
Compiling bootstrap v0.0.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/bootstrap)
Finished dev [unoptimized] target(s) in 1m 28s
Checking rustdoc artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Checking std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Compiling cc v1.0.28
Checking core v0.0.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/libcore)
Compiling libc v0.2.51
Compiling unwind v0.0.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/libunwind)
Compiling build_helper v0.1.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/build_helper)
Compiling compiler_builtins v0.1.8
Compiling cmake v0.1.33
Compiling backtrace-sys v0.1.27
Compiling std v0.0.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/libstd)
Checking rustc-std-workspace-core v1.0.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/tools/rustc-std-workspace-core)
Compiling rustc_asan v0.0.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/librustc_asan)
Compiling rustc_lsan v0.0.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/librustc_lsan)
Compiling rustc_tsan v0.0.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/librustc_tsan)
Compiling rustc_msan v0.0.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/librustc_msan)
Checking alloc v0.0.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/liballoc)
Checking rustc-demangle v0.1.10
Checking panic_abort v0.0.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/libpanic_abort)
Checking panic_unwind v0.0.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/libpanic_unwind)
Finished release [optimized] target(s) in 32.12s
Checking test artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Checking rustc_term v0.0.1
Checking getopts v0.2.17
Checking proc_macro v0.0.0 (/home/lzutao/github.com/lzutao/rust-fork/rust/src/libproc_macro)
Checking libtest v0.0.1
error[E0460]: found possibly newer version of crate `std` which `getopts` depends onest
--> /home/lzutao/.cargo/registry/src/github.com-1ecc6299db9ec823/libtest-0.0.1/lib.rs:34:5
|
34 | use getopts;
| ^^^^^^^
|
= note: perhaps that crate needs to be recompiled?
= note: the following crate versions were found:
crate `std`: /home/lzutao/github.com/lzutao/rust-fork/rust/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d26c9396879f81c2.rmeta
crate `getopts`: /home/lzutao/github.com/lzutao/rust-fork/rust/build/x86_64-unknown-linux-gnu/stage0-test/x86_64-unknown-linux-gnu/release/deps/libgetopts-267ef145c76bc717.rmeta
error: aborting due to previous error
For more information about this error, try `rustc --explain E0460`.
error: Could not compile `libtest`.
To learn more, run the command again with --verbose.
command did not execute successfully: "/home/lzutao/github.com/lzutao/rust-fork/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "check" "--target" "x86_64-unknown-linux-gnu" "-j" "1" "--release" "--manifest-path" "/home/lzutao/github.com/lzutao/rust-fork/rust/src/libtest/Cargo.toml" "--message-format" "json"
expected success, got: exit code: 101
failed to run: /home/lzutao/github.com/lzutao/rust-fork/rust/build/bootstrap/debug/bootstrap check
Build completed unsuccessfully in 0:02:23
</details> |
|
I get this error with My first build was done from a new cloned repo on an empty directory. The first build command was: |
|
@pnkfelix Any progress on this lately? Did you want to self-assign? |
This was referenced Jun 11, 2019
bors
added a commit
to rust-lang/cargo
that referenced
this issue
Jul 26, 2019
Fix some issues with absolute paths in dep-info files. There were some issues with how #7030 was handling translating paths in dep-info files. The main consequence is that when coupled with rust-lang/rust#61727 (which has not yet merged), the fingerprint would fail and be considered dirty when it should be fresh. It was joining [`target_root`](https://github.com/rust-lang/cargo/blame/1140c527c4c3b3e2533b9771d67f88509ef7fc16/src/cargo/core/compiler/fingerprint.rs#L1352-L1360) which had 3 different values, but stripping [only one](https://github.com/rust-lang/cargo/blame/1140c527c4c3b3e2533b9771d67f88509ef7fc16/src/cargo/core/compiler/mod.rs#L323). This means for different scenarios (like using `--target`), it was creating incorrect paths in some cases. For example a target having a proc-macro dependency which would be in the host directory. The solution here is to always use CARGO_TARGET_DIR as the base that all relative paths are checked against. This should handle all host/target differences. The tests are marked with `#[ignore]` because 61727 has not yet merged. This includes a second commit (which I can open as a separate PR if needed) which is an alternate solution to #7034. It adds dep-info tracking for registry dependencies. However, it tries to limit which kinds of paths it tracks. It will not track package-relative paths (like source files). It also adds an mtime cache to significantly reduce the number of stat calls (because every dependency was stating every file in sysroot). Finally, I've run some tests using this PR with 61727 in rust-lang/rust. I can confirm that a second build is fresh (it doesn't erroneously rebuild). I can also confirm that the problem in rust-lang/rust#59105 has *finally* been fixed! My confidence in all this is pretty low, but I've been unable to think of any specific ways to make it fail. If anyone has ideas on how to better test this, let me know. Also, feel free to ask questions since I've been thinking about this a lot for the past few weeks, and there is quite a bit of subtle stuff going on.
Mark-Simulacrum
added a commit
to Mark-Simulacrum/rust
that referenced
this issue
Aug 14, 2019
…xcrichton Utilize -Zbinary-dep-depinfo in rustbuild The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing). We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles. Both of these are fixable in the longer term but this existing patch gives us the following benefits: * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712. * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481. * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719. * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105. cc rust-lang#63012
Centril
added a commit
to Centril/rust
that referenced
this issue
Aug 15, 2019
…xcrichton Utilize -Zbinary-dep-depinfo in rustbuild The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing). We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles. Both of these are fixable in the longer term but this existing patch gives us the following benefits: * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712. * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481. * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719. * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105. cc rust-lang#63012
Centril
added a commit
to Centril/rust
that referenced
this issue
Aug 15, 2019
…xcrichton Utilize -Zbinary-dep-depinfo in rustbuild The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing). We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles. Both of these are fixable in the longer term but this existing patch gives us the following benefits: * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712. * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481. * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719. * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105. cc rust-lang#63012
Centril
added a commit
to Centril/rust
that referenced
this issue
Aug 15, 2019
…xcrichton Utilize -Zbinary-dep-depinfo in rustbuild The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing). We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles. Both of these are fixable in the longer term but this existing patch gives us the following benefits: * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712. * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481. * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719. * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105. cc rust-lang#63012
Centril
added a commit
to Centril/rust
that referenced
this issue
Aug 15, 2019
…xcrichton Utilize -Zbinary-dep-depinfo in rustbuild The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing). We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles. Both of these are fixable in the longer term but this existing patch gives us the following benefits: * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712. * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481. * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719. * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105. cc rust-lang#63012
Centril
added a commit
to Centril/rust
that referenced
this issue
Aug 15, 2019
…xcrichton Utilize -Zbinary-dep-depinfo in rustbuild The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing). We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles. Both of these are fixable in the longer term but this existing patch gives us the following benefits: * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712. * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481. * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719. * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105. cc rust-lang#63012
|
Just bumped onto this with |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I manage to successfully build the latest master revision of rustc, but then when I try to build it incrementally as
./x.py build -i --keep-stage 0, I get the following error. Apparently, this is broken for others as well, as has been for a while.The text was updated successfully, but these errors were encountered: