Split DWARF inlining should be disabled by default

[khuey]

I'm trying out -Csplit-debuginfo=packed on a large proprietary Rust project. I see that much of the debug info gets moved to the DWP file, but a lot gets left behind in the main executable.

e.g.

 readelf -S target/release/deps/app_server-d69c587c4cd8e14c.dwp
There are 10 section headers, starting at offset 0x294c5450:

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .debug_str_offset PROGBITS         0000000000000000  00000040
       0000000001c9022c  0000000000000000           0     0     1
  [ 2] .debug_abbrev.dwo PROGBITS         0000000000000000  01c9026c
       00000000002b7fd4  0000000000000000           0     0     1
  [ 3] .debug_info.dwo   PROGBITS         0000000000000000  01f48240
       000000001302d170  0000000000000000           0     0     1
  [ 4] .debug_loc.dwo    PROGBITS         0000000000000000  14f753b0
       000000000603c179  0000000000000000           0     0     1
  [ 5] .debug_str.dwo    PROGBITS         0000000000000000  1afb1529
       000000000e4dffe0  0000000000000000           0     0     1
  [ 6] .debug_cu_index   PROGBITS         0000000000000000  29491509
       0000000000033ea0  0000000000000000           0     0     1
  [ 7] .symtab           SYMTAB           0000000000000000  294c53b0
       0000000000000018  0000000000000018           8     1     8
  [ 8] .strtab           STRTAB           0000000000000000  294c53c8
       0000000000000001  0000000000000000           0     0     1
  [ 9] .shstrtab         STRTAB           0000000000000000  294c53c9
       0000000000000082  0000000000000000           0     0     1

vs

readelf -S target/release/app_server
There are 45 section headers, starting at offset 0x1e25b610:
[snip]
  [32] .debug_aranges    PROGBITS         0000000000000000  03260840
       0000000000442710  0000000000000000           0     0     16
  [33] .debug_info       PROGBITS         0000000000000000  036a2f50
       00000000060c92de  0000000000000000           0     0     1
  [34] .debug_abbrev     PROGBITS         0000000000000000  0976c22e
       0000000000080958  0000000000000000           0     0     1
  [35] .debug_line       PROGBITS         0000000000000000  097ecb86
       0000000003005a24  0000000000000000           0     0     1
  [36] .debug_frame      PROGBITS         0000000000000000  0c7f25b0
       0000000000000178  0000000000000000           0     0     8
  [37] .debug_str        PROGBITS         0000000000000000  0c7f2728
       0000000004ccf04b  0000000000000001  MS       0     0     1
  [38] .debug_loc        PROGBITS         0000000000000000  114c1773
       0000000000932322  0000000000000000           0     0     1
  [39] .debug_ranges     PROGBITS         0000000000000000  11df3a95
       00000000092b4bf0  0000000000000000           0     0     1
  [40] .debug_macro      PROGBITS         0000000000000000  1b0a8685
       0000000000032384  0000000000000000           0     0     1
  [41] .debug_addr       PROGBITS         0000000000000000  1b0daa09
       0000000001cc51e8  0000000000000000           0     0     1

When using split DWARF with clang the binary's .debug_info section contains compile units with a single DW_TAG_compile_unit/DW_TAG_skeleton_unit (depending on DWARF 4 vs DWARF 5) and nothing else. When using split DWARF with rustc (1.66.0) though, a lot of stuff is left behind in the binary. The size of the .debug_info section in the binary is still roughly 1/3rd the size of the .debug_info section in the .dwp.

One sample of what's left behind:

< 1><0x0000046a>    DW_TAG_subprogram
                      DW_AT_low_pc                0x007d09a0
                      DW_AT_high_pc               <offset-from-lowpc>376
                      DW_AT_name                  variant_seed<serde_json::error::Error, core::marker::PhantomData<variables::data_renderer::_::{impl#0}::deserialize::__Field>>
< 2><0x0000047b>      DW_TAG_inlined_subroutine
                        DW_AT_abstract_origin       <0x00000458>
                        DW_AT_ranges                0x00000f00
                ranges: 4 at .debug_ranges offset 3840 (0x00000f00) (64 bytes)
                        [ 0] range entry    0x007d09aa 0x007d0a99
                        [ 1] range entry    0x007d0aae 0x007d0ad1
                        [ 2] range entry    0x007d0ae7 0x007d0b18
                        [ 3] range end      0x00000000 0x00000000
                        DW_AT_call_file             0x00000003 /home/khuey/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.151/src/private/de.rs
                        DW_AT_call_line             0x000008a5
                        DW_AT_call_column           0x0000000d
< 3><0x00000488>        DW_TAG_inlined_subroutine
                          DW_AT_abstract_origin       <0x00000452>
                          DW_AT_ranges                0x00000f40
                ranges: 4 at .debug_ranges offset 3904 (0x00000f40) (64 bytes)
                        [ 0] range entry    0x007d09aa 0x007d0a99
                        [ 1] range entry    0x007d0aae 0x007d0ad1
                        [ 2] range entry    0x007d0ae7 0x007d0b18
                        [ 3] range end      0x00000000 0x00000000
                          DW_AT_call_file             0x00000007 /home/khuey/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.151/src/de/mod.rs
                          DW_AT_call_line             0x00000318
                          DW_AT_call_column           0x00000009
< 4><0x00000495>          DW_TAG_inlined_subroutine
                            DW_AT_abstract_origin       <0x0000044c>
                            DW_AT_ranges                0x00000f80
                ranges: 4 at .debug_ranges offset 3968 (0x00000f80) (64 bytes)
...

In general it all seems to be related to inline functions somehow. I assume the contents of debug_str/etc are supporting data for these DIEs.

[bjorn3]

This is probably from the standard library. Using -Zbuild-std to recompile it with -Csplit-debuginfo=packed` should work, but requires nightly.

[khuey]

That has no effect.

[khuey]

The cause of this issue is an LLVM (mis?)feature called "split dwarf inlining"[0] which is apparently for when you didn't really mean that you want separate debug info and adds back in enough debug info to do stack traces. Disabling that (and using -Zbuild-std for the compilation units that come from the standard library) does produce the expected output.

clang correctly disables this by default[1] and rustc should do the same.

[0]

rust/compiler/rustc_session/src/options.rs

Line 1573 in bf7ea0d

split_dwarf_inlining: bool = (true, parse_bool, [TRACKED],

[1] https://lists.llvm.org/pipermail/llvm-dev/2019-December/137464.html

[philipc]

backtrace-rs will need changes to support that.

[khuey]

backtrace-rs will need changes to support that.

Step 1 of that is merging rust-lang/cargo#11572 which makes split DWARF work with gdb.

After that I'll look at what we need to add to gimli/backtrace.