Revert "Work around invalid DWARF bugs for fat LTO" #95685

cbiffle · 2022-04-05T16:30:31Z

Since September, the toolchain has not been generating reliable DWARF
information for static variables when LTO is on. This has affected
projects in the embedded space where the use of LTO is typical. In our
case, it has kept us from bumping past the 2021-09-22 nightly toolchain
lest our debugger break. This has been a pretty dramatic regression for
people using debuggers and static variables. See #90357 for more info
and a repro case.

This commit is a mechanical revert of
d5de680 from PR #89041, which caused
the issue. (Note on that PR that the commit's author has requested it be
reverted.)

I have locally verified that this fixes #90357 by restoring the
functionality of both the repro case I posted on that bug, and debugger
behavior on real programs. There do not appear to be test cases for this
in the toolchain; if I've missed them, point me at 'em and I'll update
them.

rust-highfive · 2022-04-05T16:30:34Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @compiler-errors (or someone else) soon.

Please see the contribution instructions for more information.

compiler-errors · 2022-04-05T16:41:24Z

r? @pnkfelix since you self-assigned the attached issue (and I know very little about codegen)

jyn514 · 2022-04-05T16:51:02Z

@cbiffle do you think you could add tests for this in src/test/debuginfo so it doesn't regress again?

cbiffle · 2022-04-05T17:08:17Z

@cbiffle do you think you could add tests for this in src/test/debuginfo so it doesn't regress again?

Happy to give it a shot! A modification to basic-types-globals.rs looks like it'd suffice -- that test is not currently being run with lto on, and if it were, I believe it'd be failing.

Is it worth attempting to matrix the debug tests across different build settings so that there aren't omissions like this?

Since September, the toolchain has not been generating reliable DWARF information for static variables when LTO is on. This has affected projects in the embedded space where the use of LTO is typical. In our case, it has kept us from bumping past the 2021-09-22 nightly toolchain lest our debugger break. This has been a pretty dramatic regression for people using debuggers and static variables. See rust-lang#90357 for more info and a repro case. This commit is a mechanical revert of d5de680 from PR rust-lang#89041, which caused the issue. (Note on that PR that the commit's author has requested it be reverted.) I have locally verified that this fixes rust-lang#90357 by restoring the functionality of both the repro case I posted on that bug, and debugger behavior on real programs. There do not appear to be test cases for this in the toolchain; if I've missed them, point me at 'em and I'll update them.

cbiffle · 2022-04-05T17:43:10Z

So, there were some tests that would have caught this, except that (1) they were "temporarily" disabled in 2018 and (2) they were using the wrong build configuration. Altering basic-types-globals.rs by turning it back on and enabling LTO produces a test that fails on current master, for two reasons:

Because its assertions about GDB's output appear to have rotted in the four years it's been disabled, and
Because of the bug described in DWARF info for static vars in lib crates has stopped being produced reliably in LTO builds #90357

I've added a commit to this PR that re-enables the test and adds a duplicate of it using LTO. Both tests pass with this PR; the LTO test fails without it, reproducing the bug.

I admit I don't understand the circumstances that led to these tests being disabled in #47155, so I don't know for sure that simply turning them back on and fixing the decay is the right answer.

Also, if there is a way to matrix a single test case in multiple build configurations rather than copy-pasta, please let me know.

cbiffle · 2022-04-05T18:29:06Z

Also, for what it's worth, I've managed to update and re-enable 16 debuginfo tests, some simply by turning them back on, some by updating rotten assertions about GDB's output. It looks like there's been some regression in lldb's output in some cases, unrelated to rustc's DWARF generation, so in some cases I've left them disabled for lldb. If there's interest in this change I can create a followup PR.

jyn514 · 2022-04-05T18:30:35Z

If there's interest in this change I can create a followup PR.

That sounds amazing!

compiler-errors · 2022-04-05T18:33:05Z

Also, if there is a way to matrix a single test case in multiple build configurations rather than copy-pasta, please let me know.

@cbiffle: Look at revisions, though you might need to check out other tests for best examples for how to use it.

cbiffle · 2022-04-05T18:50:47Z

@cbiffle: Look at revisions, though you might need to check out other tests for best examples for how to use it.

Slick. That's almost exactly what I wanted. Unfortunately, any attempt to use revisions in a gdbtest hits a panic: https://github.com/rust-lang/rust/blob/master/src/tools/compiletest/src/runtest.rs#L758

- Re-enabled basic-types-globals which has been disabled since 2018 - Updated its now-rotted assertions about GDB's output to pass - Rewrote header comment describing some previous state of GDB behavior that didn't match either the checked-in assertions _or_ the current behavior, and so I assume has just been wrong for some time. - Copy-pasta'd the test into a version that uses LTO to check for regression on rust-lang#90357, because I don't see a way to matrix the same test into several build configurations.

cbiffle · 2022-04-05T19:37:53Z

If there's interest in this change I can create a followup PR.

That sounds amazing!

Posted as a draft in #95696 . It's based on top of this PR for my convenience. If it seems like this PR is going to be controversial I can rebase and separate them.

cbiffle · 2022-04-11T20:33:59Z

Hi! Is anybody looking at this? Can I do anything to facilitate? Thanks!

pnkfelix · 2022-04-14T13:31:36Z

Hi! Is anybody looking at this? Can I do anything to facilitate? Thanks!

Hi @cbiffle , I'm looking at it now. Hopefully will merge soon.

pnkfelix · 2022-04-14T13:33:30Z

This commit is a mechanical revert ofhttps://github.com/rust-lang/rust/commit/d5de680e20def848751cb3c11e1182408112b1d3 from PR #89041, which caused
the issue. (Note on that PR that the commit's author has requested it be
reverted.)

This is pretty easy justification for a revert here. :)

Thanks @cbiffle !

@bors r+ rollup

bors · 2022-04-14T13:33:31Z

📌 Commit 42af197 has been approved by pnkfelix

… r=pnkfelix Revert "Work around invalid DWARF bugs for fat LTO" Since September, the toolchain has not been generating reliable DWARF information for static variables when LTO is on. This has affected projects in the embedded space where the use of LTO is typical. In our case, it has kept us from bumping past the 2021-09-22 nightly toolchain lest our debugger break. This has been a pretty dramatic regression for people using debuggers and static variables. See rust-lang#90357 for more info and a repro case. This commit is a mechanical revert of d5de680 from PR rust-lang#89041, which caused the issue. (Note on that PR that the commit's author has requested it be reverted.) I have locally verified that this fixes rust-lang#90357 by restoring the functionality of both the repro case I posted on that bug, and debugger behavior on real programs. There do not appear to be test cases for this in the toolchain; if I've missed them, point me at 'em and I'll update them.

GrantM11235 · 2022-04-30T18:32:30Z

What is the status of this PR? Does bors need to try again?

jyn514 · 2022-05-01T17:39:24Z

@GrantM11235 it failed the multiplatform tests - someone (presumably @cbiffle, although other people can take it over if they don't have time) needs to figure out why and fix the tests.

cbiffle · 2022-05-17T20:21:26Z

@jyn514 I'd like to help however I can, but I don't understand the failure these hit.

cbiffle · 2022-06-01T19:35:20Z

@pnkfelix or anyone else, do you understand the build failure here? We're getting increasingly far behind the rust-embedded ecosystem because they're using features that became available after DWARF generation broke, whereas we require DWARF generation. I'd like to do what I can to move this along but my fix works on the platforms I have access to.

nikic · 2022-06-01T21:23:05Z

The build failure is because the newly reenabled tests fail. It looks like #95138 also tried to reenable gdb tests, but also ran into failures. Most likely, these don't work under gdb 7, but work under some newer version. You could try adding a min-gdb-version annotation.

JohnCSimon · 2022-07-03T01:45:57Z

Ping from triage:
@cbiffle - can you post your status on this PR?

luqmana · 2022-07-15T20:56:28Z

@rustbot ready

Spoke w/ @cbiffle to help get this landed.

I updated the test to include a minimum GDB version. The CI job linked above that failed was using 7.11.1-0ubuntu1~16.5. I did confirm that it also fails locally with 7.12.1 but passes starting with 8.0.

While this does mean it won't be run on some of the CI machines, we already have a number of tests that specify a minimum GDB version of anywhere from 7.7 to 10.1. This should still give us some coverage (e.g. src/test/debuginfo/rc_arc.rs is marked as min-gdb-version: 8.1 and is successfully run as part of auto (x86_64-gnu-stable, x86_64-gnu, stable, nightly, ubuntu-20.04-xl)).

jyn514 · 2022-07-15T21:28:54Z

@bors r=pnkfelix rollup=never

Thanks!

bors · 2022-07-15T21:28:55Z

📌 Commit 2085d6a has been approved by pnkfelix

It is now in the queue for this repository.

bors · 2022-07-15T21:29:07Z

⌛ Testing commit 2085d6a with merge 1d91b6bba8269b811ffc115121616b3d0d2ef43f...

Mark-Simulacrum · 2022-07-15T21:37:56Z

@bors retry prioritizing #99288

rust-log-analyzer · 2022-07-15T21:49:07Z

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

bors · 2022-07-16T00:18:58Z

⌛ Testing commit 2085d6a with merge e6c43cf...

bors · 2022-07-16T02:59:34Z

☀️ Test successful - checks-actions
Approved by: pnkfelix
Pushing e6c43cf to master...

rust-timer · 2022-07-16T04:15:54Z

Finished benchmarking commit (e6c43cf): comparison url.

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

Primary benchmarks: 😿 relevant regressions found
Secondary benchmarks: 😿 relevant regression found

	mean¹	max	count²
Regressions 😿 (primary)	3.2%	3.6%	2
Regressions 😿 (secondary)	2.9%	2.9%	1
Improvements 🎉 (primary)	N/A	N/A	0
Improvements 🎉 (secondary)	N/A	N/A	0
All 😿🎉 (primary)	3.2%	3.6%	2

Cycles

Results

Primary benchmarks: 🎉 relevant improvement found
Secondary benchmarks: 🎉 relevant improvement found

	mean¹	max	count²
Regressions 😿 (primary)	N/A	N/A	0
Regressions 😿 (secondary)	N/A	N/A	0
Improvements 🎉 (primary)	-3.4%	-3.4%	1
Improvements 🎉 (secondary)	-2.7%	-2.7%	1
All 😿🎉 (primary)	-3.4%	-3.4%	1

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

the arithmetic mean of the percent change ↩ ↩²
number of relevant changes ↩ ↩²

compiletest: Allow using revisions with debuginfo tests. A small wart that came up in rust-lang#95685 (comment).

rust-highfive assigned compiler-errors Apr 5, 2022

rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Apr 5, 2022

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 5, 2022

rust-highfive assigned pnkfelix and unassigned compiler-errors Apr 5, 2022

cbiffle force-pushed the restore-static-dwarf branch from 68800da to 3f48e92 Compare April 5, 2022 17:38

cbiffle force-pushed the restore-static-dwarf branch from 3f48e92 to 42af197 Compare April 5, 2022 19:32

cbiffle mentioned this pull request Apr 5, 2022

Reenable debuginfo tests #95696

Closed

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 14, 2022

fee1-dead mentioned this pull request Apr 15, 2022

Rollup of 16 pull requests #96069

Closed

RalfJung mentioned this pull request Apr 15, 2022

Rollup of 8 pull requests #96075

Closed

Set minimum GDB version needed for basic-types-globals* tests.

2085d6a

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 15, 2022

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 15, 2022

bors added the merged-by-bors This PR was explicitly merged by bors. label Jul 16, 2022

bors merged commit e6c43cf into rust-lang:master Jul 16, 2022

rustbot added this to the 1.64.0 milestone Jul 16, 2022

bors mentioned this pull request Jul 16, 2022

Re-enable some gdb tests #95138

Closed

steveklabnik mentioned this pull request Jul 18, 2022

bump toolchain to last night's nightly oxidecomputer/hubris#438

Merged

luqmana mentioned this pull request Jul 20, 2022

compiletest: Allow using revisions with debuginfo tests. #99488

Merged

matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Jul 20, 2022

Rollup merge of rust-lang#99488 - luqmana:debuginfo-revisions, r=tmiasko

a0c6964

compiletest: Allow using revisions with debuginfo tests. A small wart that came up in rust-lang#95685 (comment).

luqmana deleted the restore-static-dwarf branch September 23, 2022 21:36

jieyouxu mentioned this pull request Aug 14, 2024

Tracking Issue for Reproducible Build bugs and challenges #129080

Open

81 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Revert "Work around invalid DWARF bugs for fat LTO" #95685

Revert "Work around invalid DWARF bugs for fat LTO" #95685

cbiffle commented Apr 5, 2022

rust-highfive commented Apr 5, 2022

compiler-errors commented Apr 5, 2022

jyn514 commented Apr 5, 2022

cbiffle commented Apr 5, 2022

cbiffle commented Apr 5, 2022 •

edited

Loading

cbiffle commented Apr 5, 2022

jyn514 commented Apr 5, 2022

compiler-errors commented Apr 5, 2022

cbiffle commented Apr 5, 2022

cbiffle commented Apr 5, 2022

cbiffle commented Apr 11, 2022

pnkfelix commented Apr 14, 2022

pnkfelix commented Apr 14, 2022

bors commented Apr 14, 2022

GrantM11235 commented Apr 30, 2022

jyn514 commented May 1, 2022

cbiffle commented May 17, 2022

cbiffle commented Jun 1, 2022

nikic commented Jun 1, 2022

JohnCSimon commented Jul 3, 2022

luqmana commented Jul 15, 2022 •

edited

Loading

jyn514 commented Jul 15, 2022

bors commented Jul 15, 2022

bors commented Jul 15, 2022

Mark-Simulacrum commented Jul 15, 2022

rust-log-analyzer commented Jul 15, 2022

bors commented Jul 16, 2022

bors commented Jul 16, 2022

rust-timer commented Jul 16, 2022

Revert "Work around invalid DWARF bugs for fat LTO" #95685

Revert "Work around invalid DWARF bugs for fat LTO" #95685

Conversation

cbiffle commented Apr 5, 2022

rust-highfive commented Apr 5, 2022

compiler-errors commented Apr 5, 2022

jyn514 commented Apr 5, 2022

cbiffle commented Apr 5, 2022

cbiffle commented Apr 5, 2022 • edited Loading

cbiffle commented Apr 5, 2022

jyn514 commented Apr 5, 2022

compiler-errors commented Apr 5, 2022

cbiffle commented Apr 5, 2022

cbiffle commented Apr 5, 2022

cbiffle commented Apr 11, 2022

pnkfelix commented Apr 14, 2022

pnkfelix commented Apr 14, 2022

bors commented Apr 14, 2022

GrantM11235 commented Apr 30, 2022

jyn514 commented May 1, 2022

cbiffle commented May 17, 2022

cbiffle commented Jun 1, 2022

nikic commented Jun 1, 2022

JohnCSimon commented Jul 3, 2022

luqmana commented Jul 15, 2022 • edited Loading

jyn514 commented Jul 15, 2022

bors commented Jul 15, 2022

bors commented Jul 15, 2022

Mark-Simulacrum commented Jul 15, 2022

rust-log-analyzer commented Jul 15, 2022

bors commented Jul 16, 2022

bors commented Jul 16, 2022

rust-timer commented Jul 16, 2022

Instruction count

Max RSS (memory usage)

Cycles

Footnotes

cbiffle commented Apr 5, 2022 •

edited

Loading

luqmana commented Jul 15, 2022 •

edited

Loading