Missing backtraces in local compiler build

[davidtwco]

I recently rebased an old branch (branched off at 4007d4e) and afterwards found that the backtrace didn't contain any of the compiler (ignore the actual ICE, I'm sure I'll fix it once I stop trying to get backtraces back):

error: internal compiler error: src/librustc/traits/codegen/mod.rs:58: Encountered error `Unimplemented` selecting `Binder(<T as num::dec2flt::rawfp::RawFloat>)` during codegen

thread 'rustc' panicked at 'Box<Any>', src/librustc_errors/lib.rs:895:9
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /home/david/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /home/david/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:77
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:59
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1057
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1426
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:62
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:49
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:195
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:215
error: could not compile `core`.

You can find my config.toml here. This is running on a NixOS host with an i7-8700. It isn't fixed by setting RUST_BACKTRACE to 1, full or leaving it unset. I've also tried with and without the backtrace option in config.toml. I also tried invoking the compiler directly (original error was in compiling libcore through x.py after some of my changes).

I was able to bisect (results) the issue to a handful of commits:

• 150d328
• 9245182
• a7de090
• 8b9c539
• 1aa34d4
• fca192c
• f94b0e6
• 91b25a8
• 7f23e6e

I wasn't able to test any of these and be more specific because Cargo.lock was in a broken state in these commits. On the last commit which failed, after printing the same limited backtrace as above (on a simple test case that compile errors and -Z treat-err-as-bug), it printed the following:

   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:193 
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:210    
fish: “rustc +rust2-stage1 -Z treat-er…” terminated by signal SIGSEGV (Address boundary error)

cc @Aaron1011 @alexcrichton (authors of the above commits)

[davidtwco]

I've found that I can also reproduce this in a Ubuntu 18.04 LXC container on the same host. I also noticed that the command that failed mentions which signal caused the failure:

  process didn't exit successfully: `/working/build/bootstrap/debug/rustc --edition=2018 --crate-name core src/libcore/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=2 -C debuginfo=2 -C metadata=48803f0c286bf5f4 -C extra-filename=-48803f0c286bf5f4 --out-dir /working/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps --target x86_64-unknown-linux-gnu -C incremental=/working/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/incremental -L dependency=/working/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps -L dependency=/working/build/x86_64-unknown-linux-gnu/stage1-std/release/deps -Zexternal-macro-backtrace '-Clink-args=-Wl,-rpath,$ORIGIN/../lib' -Wrust_2018_idioms -Wunused_lifetimes -Dwarnings -Cprefer-dynamic -Zbinary-dep-depinfo` (signal: 11, SIGSEGV: invalid memory reference)

[davidtwco]

Changing the kernel version from 4.19.88 to 5.4.2 doesn't appear to have an effect (not that I would particularly have expected it to).

[davidtwco]

With some more experimentation, I've found that this is a result of debug = true or debuginfo-level > 1 (it also changed the ICE that I was seeing with my local changes, strangely).

[davidtwco]

I've rebased and this has started happening again, but now with any debuginfo-level.

error: internal compiler error: src/librustc/ty/instance.rs:85: Instance.ty called for type fn() -> u32 {closures::unused::<T>} with params in substs: [T]

thread 'rustc' panicked at 'Box<Any>', src/librustc_errors/lib.rs:885:9
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /home/david/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/libunwind.rs:86
   1: backtrace::backtrace::trace_unsynchronized
             at /home/david/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:78
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:59
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1053
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1428
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:62
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:49
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:204
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:224
fish: Process 25930, “rustc” “rustc +rust0-stage1 src/test/co…” terminated by signal SIGSEGV (Address boundary error)

[tmiasko]

Potentially the same as #69151, which has more details regarding root cause.

[davidtwco]

@tmiasko thanks, closing this in favour of #69151.