migrate to the LLVM toolchain #9367
Comments
|
nominating. At the least we need to switch our linker away from gcc to something else on windows in order to get away from requiring a mingw install just to run rustc. |
|
1.0, high. Windows needs to work. |
|
I was doing some testing with this recently, and here's all the troubles I ran into:
I have done little testing on windows, but I believe that the story will be very similar except for figuring out where |
|
FWIW, I've checked out a recent Windows build of lld (from llvm.org/builds), and it could not link objects produced by mingw:
|
|
@brson, @alexcrichton: Looks like I've succeeded in compiling compiler-rt on mingw. (I hope other platforms will be easier because it was actually intended to be built there). This opens up 2 possible venues for further work:
|
|
cc #749 |
|
Interesting! I think there's no question about moving to compiler-rt instead of libgcc. I'd be more than willing to help you integrate it with the current build system! How does using compiler-rt instead of libgcc relate to valgrind vs address sanitizer? |
|
Address sanitizer runtime is a part of compiler-rt. Rust could use compiler-rt just for that, for starters. |
|
Hm, if we could not compile the asan part of compiler-rt for now I think that may be the best way to go. Depending on libgcc is fine for unwinding, moving to compiler-rt is the motivation for fixing bugs like #8449. |
|
asan compiles just fine actually. But using it will require some plumbing in rustc, such as emitting sanitize_address attributes on functions and running asan pass in llvm. ok I'll look into migration from libgcc. |
|
Looks like this is essentially done except for |
|
Nominating for removal from the 1.0 milestone. I don't think there's much we can do about this except "wait and see" |
|
Agreed, the feasible part is already finished. |
|
Since work on this is gated on Removing from 1.0 milestone, and demoting to P-low (since the high priority items have now been addressed). |
|
Migrating away from MinGW completely is actually within the realm of possibility. |
|
Rust can't redistribute link.exe and it's still an extra executable to spawn so it would be a step backwards from MinGW-w64, not progress. The current toolchain is open-source and be redistributed. It works fine with MSVC C libraries and Rust can't bind directly to C++ anyway. |
|
As shown in IRC earler, LLD is now capable of dynamically linking Rust programs. |
sanxiyn
added
the
A-LLVM
|
I briefly explored getting rustc built with LLD today, but found the current linker situation complicated enough to discourage me from further work. In summary:
Note that LLD intentionally implements several command-line interfaces, designed to emulate There's a final snag:
This means that in order to use LLD, rustc must depend on If anyone is interested in picking up this torch (maybe @alexcrichton or @alexchandel?), see https://github.com/tamird/rust/tree/use-lld. |
|
@tamird do you just need help getting lld integrated into our build system? Or do you need help getting it integrated into the compiler itself? You may be able to get by just installing lld into your |
|
@alexcrichton yep, that's what I've done for now: A bunch of tests are passing (but some are failing), so:
EDIT: test run just finished, all the failures are: |
|
@alexcrichton here's the full output: https://gist.github.com/tamird/88215c54395e29bd7d8e |
|
@tamird Was that before or after @alexcrichton's change to unwinding? The one that made |
|
@eddyb that exact error happens on a snapshot (rustc 1.4.0-nightly (d503524 2015-08-29)). See repro instructions https://github.com/tamird/rust_lld EDIT: those instructions include running |
|
Microsoft has already opensource it's PDB/CodeView format and preparing implemented it in LLVM, that's would be a great news for lld. |
|
My experience using I started with a hello world program: Then produced an object file and got the linker arguments used by These arguments don't work with
I ended with a target=x86_64-unknown-linux-gnu
sysroot=$(rustc --print sysroot)
libdir=${sysroot}/lib/rustlib/${target}/lib
lld \
-flavor gnu \
-L /usr/lib \
--as-needed \
-z,noexecstack \
-L ${libdir} \
hello.0.o \
/usr/lib/crt1.o \
/usr/lib/crti.o \
-o hello \
--gc-sections \
-Bstatic \
-Bdynamic \
${libdir}/libstd-fd663c41.rlib \
${libdir}/libcollections-fd663c41.rlib \
${libdir}/librustc_unicode-fd663c41.rlib \
${libdir}/librand-fd663c41.rlib \
${libdir}/liballoc-fd663c41.rlib \
${libdir}/liballoc_jemalloc-fd663c41.rlib \
${libdir}/liblibc-fd663c41.rlib \
${libdir}/libcore-fd663c41.rlib \
-l dl \
-l pthread \
-l gcc_s \
-l c \
-l m \
-l rt \
-l compiler-rtThen I got the following segfault with the above Interestingly removing the Other interesting bit is that VersionCan anyone reproduce these errors? Or perhaps someone can point out a fatal error I may have committed in the above steps? |
|
ELF |
|
Aha, that would explain the errors. I'll try again at a later date 😄. |
|
FWIW, I've updated https://github.com/tamird/rust_lld, and things seem to work as before against LLVM HEAD ( That is, "hello world" works, but panic doesn't (illegal instruction). I'm not sure what you were doing wrong @japaric but I'd be happy to collaborate if you want to use my work as a starting point. |
|
@nagisa It will be fixed eventually right? It'd be a pity to only be able to link mach-o and windows binaries. FWIW, by having a shell script called |
|
@alexchandel they’re rewriting ELF backend from scratch. |
|
@tamird I think Anyhow, I intended to add a |
You must be using GCC; Also, note that some of the scripts invoke |
|
Just saying, if we had #30027 then we wouldn't depend on import libraries from mingw/msvc on Windows. |
|
@retep998 This would mean that, using LLD, you could target windows without a toolchain right? Or would you still need a single |
|
If you don't care about statically linking with C/C++ code then you can easily write your own entry point and skip CRT initialization entirely. |
|
@retep998 So we could link against @nagisa They've finished rewriting and are working on feature parity now. |
I’m not sure I would call that state finished yet, but sure. |
|
@alexchandel How is that Microsoft getting it wrong? Is there a reason that statically linking the entry point might be a bad idea? On Windows every binary (exe or dll) has an entry point which is responsible for initializing all the statics and such in that binary. If you're using C/C++ in a given binary, then you need the CRT entry point. Since Rust doesn't have runtime static initializers, or any life outside of main for that matter, as long as you don't call certain CRT functions (such as Whether we link to msvcrt via generated idata or an import library is rather irrelevant to whether we skip the CRT entry point. As long as the CRT is dynamically linked and we don't have any statically linked C/C++ then we can skip the CRT entry point. Note that generating idata instead of linking to an import library is a move that needs to be considered very carefully. Imagine if we generated idata to some of the math functions and someone statically linked in some C/C++ code that calls those math functions and checks the result via |
|
@retep998 There's more than just the entry point and static initializers in there, and the versioned It sounds like Rust would have to reason whether any import libraries were included when deciding to generate idata. But in the case where C/C++ code is statically linked, a use case I often deal with on Windows, is it then necessary to use import libraries from a Windows toolchain? |
|
Update: #36120 is a PoC PR that embeds lld into rustc itself. It has only been tested with x86_64 ELFs though. |
|
This is a pretty old bug at this point and the only remaining task, lld, is best tracked in a future issue (if necessary) |
|
Is it still the intention to eventually switch to using lld? |
|
If it makes sense to, sure! |
|
I've filed #39915 for tracking LLD integration. |
and others
Especially on Windows, we shouldn't have a dependency on either MinGW or MSVC++. LLVM's toolchain is not entirely mature, but they do have C/C++ working on Windows independently of other toolchains.
http://blog.llvm.org/2013/09/a-path-forward-for-llvm-toolchain-on.html
libgcc -> compiler-rt (required to fix some other bugs like 64-bit signed/unsigned overflow checked multiplication unavailable on 32-bit #8449)(closed by compiler-rt #12027)libsupc++ (exception support) -> libc++abi (may still have a dependency on libgcc on some platforms)(closed by Remove C++ dependencies #11121)ship clang with snapshot to build the C runtime bits, and drop the C++ bits (to avoid needing to build libc++ as a replacement for libstdc++ too)(closed by 508b7b9)The text was updated successfully, but these errors were encountered: