rustc_driver.dll size on x86_64-pc-windows-gnu toolchain

[davidhewitt]

I noticed that the size of rustc_driver.dll for the windows x86_64-pc-windows-gnu toolchain seems suprisingly large compared to the x86-64-pc-windows-msvc toolchain. It's also been growing quickly in recent releases.

See chart below comparing the size of rustc_driver.dll for these two targets for stable releases since 1.40:

Some size difference is obviously expected, however the huge jump around Rust 1.54 (slightly over doubling of the file size) looks suspect.

Bisecting on nightlies around that time, it looks like the jump happened in nightly 2021-06-05:

nightly	x86_64-pc-windows-gnu rustc_driver size (bytes)
2021-06-03	199015985
2021-06-04	199027556
2021-06-05	442938696
2021-06-06	442876820

I've recently been working in an environment where the file size matters. I don't know much about the compiler packaging process, however I'd be happy to help investigate and (if possible) undo this jump in file size, with the help of someone willing to guide me where to look.

So far, I started out by looking at PRs merged on 2021-06-04 (i.e. the day before the size bump).

The only PR that stands out to me as potentially affecting file size is #84449. I am wildly speculating; this could be a red herring. I'm going to investigate this a little further. Input from others more knowledgeable than myself is very welcome!

[davidhewitt]

It looks like #84449 might well be relevant. Here's some outputs from dumpbin:

First, nightly 2021-06-04 for the gnu toolchain:

PS C:\Users\david\Dev\rust-size> dumpbin.exe C:\Users\david\.rustup\toolchains\nightly-2021-06-04-x86_64-pc-windows-gnu\bin\rustc_driver-dec9162dd471bbf9.dll
Microsoft (R) COFF/PE Dumper Version 14.26.28806.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file C:\Users\david\.rustup\toolchains\nightly-2021-06-04-x86_64-pc-windows-gnu\bin\rustc_driver-dec9162dd471bbf9.dll

File Type: DLL

  Summary

        1000 .CRT
        2000 .bss
       B6000 .data
        1000 .debug_abbrev
        1000 .debug_aranges
        1000 .debug_frame
       30000 .debug_info
       18000 .debug_line
        1000 .debug_loc
       1A000 .debug_pubnames
        1000 .debug_pubtypes
       40000 .debug_ranges
       52000 .debug_str
      1BA000 .edata
        1000 .eh_frame
       11000 .idata
      1CC000 .pdata
     1B69000 .rdata
       90000 .reloc
       26000 .rustc
     5384000 .text
        1000 .tls
      3D4000 .xdata

Second, nightly 2021-06-05 for the gnu toolchain. Note that the output is very similar, except for a new .rmeta section. I'm not an expert on dumpbin output; if I understand the size of that section E87F000 is a hex size in bytes, that converts to a decimal size of about 240 MB, which would almost exactly explain the size jump.

PS C:\Users\david\Dev\rust-size> dumpbin.exe C:\Users\david\.rustup\toolchains\nightly-2021-06-05-x86_64-pc-windows-gnu\bin\rustc_driver-3fd3f7c94716c1a7.dll
Microsoft (R) COFF/PE Dumper Version 14.26.28806.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file C:\Users\david\.rustup\toolchains\nightly-2021-06-05-x86_64-pc-windows-gnu\bin\rustc_driver-3fd3f7c94716c1a7.dll

File Type: DLL

  Summary

        1000 .CRT
        2000 .bss
       B6000 .data
        1000 .debug_abbrev
        1000 .debug_aranges
        1000 .debug_frame
       30000 .debug_info
       18000 .debug_line
        1000 .debug_loc
       1A000 .debug_pubnames
        1000 .debug_pubtypes
       40000 .debug_ranges
       52000 .debug_str
      1B9000 .edata
        1000 .eh_frame
       11000 .idata
      1CC000 .pdata
     1B6E000 .rdata
       90000 .reloc
     E87F000 .rmeta
       26000 .rustc
     5395000 .text
        1000 .tls
      3D6000 .xdata

cc @alexcrichton - is it possible that the "tricks" referenced in #84449 to remove the .rmeta section is not working correctly for the windows gnu toolchain?

The only caveat we
have to take care of is to ensure that the linker never actually
includes the contents of the object file into the final output. We apply
similar tricks as the .llvmbc bytecode sections to do this.

[nikic]

This would probably be fixed by #92107. There was a typo that prevented actually setting the necessary flag.

Edit: Actually, not sure on that. I think that would affect the size for the msvc target, not the gnu target, so the opposite of what is described here.

[alexcrichton]

Oh dear I apologize for the bug here! It was definitely an oversight on my part to let that typo through.

IIRC I tested at least on MSVC that the flag was respected, but given the graph in this issue and the fact that the typo flew under the radar it seems the MSVC link.exe linker doesn't need this flag to be set to ensure that .rmeta doesn't make it into binaries. I don't remember whether I tested *-windows-gnu but I probably didn't. I agree with @nikic in that I'm not sure whether #92107 will fix this but it would be good to double-check after that lands to see what's going on. Otherwise there may be some other MinGW-linker-specific thing that can be done here.

[davidhewitt]

Thanks both - I see #92107 is about to be merged in a rollup, so I'll check the next nightly produced after that lands and report back here.

[davidhewitt]

Following up on this - it looks like #92107 has fixed this.

Output of dumpbin with today's nightly no longer shows the .rmeta section:

dumpbin C:\Users\david\.rustup\toolchains\nightly-2022-01-05-x86_64-pc-windows-gnu\bin\rustc_driver-936cadc722c56399.dll
Microsoft (R) COFF/PE Dumper Version 14.29.30138.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file C:\Users\david\.rustup\toolchains\nightly-2022-01-05-x86_64-pc-windows-gnu\bin\rustc_driver-936cadc722c56399.dll

File Type: DLL

  Summary

        1000 .CRT
        2000 .bss
       A5000 .data
        1000 .debug_abbrev
        1000 .debug_aranges
        1000 .debug_frame
       3F000 .debug_info
       18000 .debug_line
        1000 .debug_loc
       2D000 .debug_pubnames
        1000 .debug_pubtypes
       32000 .debug_ranges
       6A000 .debug_str
      18B000 .edata
        1000 .eh_frame
       10000 .idata
      1E2000 .pdata
     1D3F000 .rdata
       90000 .reloc
       34000 .rustc
     58E1000 .text
        1000 .tls
      416000 .xdata

rustc_driver.dll size is now back at ~230MB, which seems about right when comparing against nightlies back in May 2021.

Thanks @nikic for pushing a bugfix before I even reported this 😂

[alexcrichton]

Nice! Apologies again on the oversight and thanks @nikic for fixing it!