-
-
Notifications
You must be signed in to change notification settings - Fork 13.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gccCrossStageStatic: enable dynamic libraries, rename it #238154
Conversation
I'm testing builds on a pretty large matrix of target platforms and gcc versions. In spite of this, I have the nagging feeling that this is going to break some obscure gcc version on an infrequently-used platform. I've tried to keep this PR
|
This comment was marked as outdated.
This comment was marked as outdated.
Everything tests fine, rebooted the laptop with a complete system (bootloader, kernel, userspace) built this way.
Edit: what I wrote in the previous paragraph is true, but I will do that in a separate PR after this one. The only thing we need gccWithoutTargetLibc to do is compile Getting |
Result of 2 packages blacklisted:
50 packages failed to build:
10 packages built:
|
(Edit) Most (maybe all) of this is fixed by these two PRs (mostly the second one):
(edit) So this doesn't happen again: |
…tic-undo-the-hack gcc: revert "kludge to prevent mass-rebuild" from #238154
Bisect claims that one of these commits also broke
I think |
And bisect claims that one of these commits also broke
Don't see why immediately. |
Please add these to Also, I'm pretty sure none of us can afford S390 hardware. |
|
@ofborg build tests.cross.sanity |
Added |
### Summary This PR completely and finally solves the gcc<->glibc circular `buildInputs` problem, for cross compilation. The same technique can be applied to native builds in the future. Closes NixOS#213453 ### Motivation Prior to this PR, we had the following circular `buildInputs` problem: 1. gcc has glibc in its `buildInputs` - a compiled copy of glibc must be present before building gcc; if it isn't, gcc cripples itself (`inhibit_libc`) and refuses to build libgcc_s.so 2. glibc has libgcc_s.so in its `buildInputs` - glibc `dlopen()`s libgcc_s.so in order to implement POSIX thread cancellation. For security reasons `glibc` requires that the path to `libgcc_s.so` is [hardwired] into `glibc` at compile time, so it's technically not a true dynamic link -- it just pretends to be one. 3. libgcc_s.so is built in the same derivation as gcc - libgcc_s.so is built as part of the gcc build process We must cut one of these three links in the loop. ### Previous Attempts Previously NixOS#238154 had attempted to cut link (1) by building `gcc` without `glibc`, and using the `libgcc_s` which emerges from that build. Unfortunately this just doesn't work. GCC's configure script extracts quite a lot of information from the glibc headers (which are a build artifact -- you can't just copy them out of the source tarball) and various `./configure`-driven linking attempts. If `glibc` isn't around at build time you wind up with a `libgcc_s.so` that is missing various unwinder features (see NixOS#213453 for the most problematic one). Musl "cuts" link (2), or rather never creates it in the first place. ["Cancellation cleanup handling in musl has no relationship to C++ exceptions and unwinding... glibc implements cancellation as an exception"](https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-cancellation). IMHO Musl made the smarter decision here. It is incredibly rare to find a codebase that uses both POSIX thread cancellation *and* C++ exceptions. I have never seen a codebase that uses both *and* expects them to be aware of each other, and I would be astonished if one existed. Glibc paid an immense cost in complexity for something nobody has ever used. ### Changes Made This PR cuts link (3): instead of building libgcc_s.so as part of gcc, we build it separately from gcc. Now there is a strict acyclic graph of `buildInputs`: ``` gccWithoutTargetLibc | +--->glibc-nolibgcc | | | v +--->libgcc | | | v +--->glibc | | | v +--->gcc ``` In other words, there's a simple linear `buildInputs` chain `glibc-nolibgcc` `->` `libgcc` `->` `glibc` `->` `gcc` where all four packages are compiled by (and therefore have as a `(native)BuildInput`) `gccWithoutTargetLibc`. `gccWithoutTargetLibc` and `glibc-nolibgcc` are strictly bootstrapping artifacts; nothing else has them as a `buildInput` and they shouldn't appear in the closure of any final deployment packages. `glibc-nolibgcc` lacks `libgcc_s.so`, so it will segfault if you try to use it with POSIX thread cancellation. Fortunately all we need from it is (a) its headers (`lib.getDev`) and (b) to use it in the `./configure` script for `libgcc`. When translated over to the native bootstrap, `xgcc` takes the place of `gccWithoutTargetLibc`, and the "first `glibc`" (we build two of them) takes the place of `glibc-nolibgcc`. At that point our native and cross bootstrap have the same overall architecture, and it becomes possible to merge them (at last!) [213453]: NixOS#213453 [238154]: NixOS#238154 [hardwired]: https://github.com/NixOS/nixpkgs/blob/7553d0fe29801938bcb280bb324b579ef9016aea/pkgs/development/libraries/glibc/default.nix#L69-L88
Another regression is |
I'm guessing the automatic backport failed and we never got backport to 23.05? Could this be backported, it would be nice, as I'm seeing the cross-compiled Wayland failing on runtime, and this PR fixes the issue |
Backport failed for Please cherry-pick the changes locally. git fetch origin release-23.05
git worktree add -d .worktree/backport-238154-to-release-23.05 origin/release-23.05
cd .worktree/backport-238154-to-release-23.05
git checkout -b backport-238154-to-release-23.05
ancref=$(git merge-base 931b8a4979df477c0baffc2544f593d7a9b32dcb 96a2f1b4e1315251f1916f64c4632dbf7eb40522)
git cherry-pick -x $ancref..96a2f1b4e1315251f1916f64c4632dbf7eb40522 |
This PR caused some regressions, so any potential backport should probably be okayed by the release managers before being merged. |
Best reviewed one commit at a time.
Motivation
Fix a serious bug in our
pkgsCross
bootstrap: cross compiled binaries segfault on pthread_cancel(3) due to missing libgcc_s.so #240484. This is the reason for the backport request.a. Also: Closes libgcc_s.so is missing from glibc when cross compiling #40797
Make our cross-compiler bootstrap look more like our
stdenv
bootstrap, as a step towards unifying them.Summary
This PR allows
gccCrossStageStatic
to build dynamically-linked libraries. Since is no longer restricted to building static libraries its name is no longer appropriate, and this PR also renames it to the more-accurategccWithoutTargetLibc
.Detailed description of changes
By default, you can't build a gcc that knows how to create dynamic libraries unless you have already built the targetPlatform libc.
Because of this, our gcc cross-compiler is built in three stages:
Build a cross-compiler (gccCrossStageStatic) that can build only static libraries.
Use gccCrossStageStatic to compile the targetPlatform
libc.a
.Use the targetPlatform
libc.a
to build a fully-capable cross compiler.You might notice that this pattern looks very similar to what we do with
xgcc
andglibc
in the stdenv bootstrap. Indeed it is! I would like to work towards getting the existing stdenv bootstrap to handle cross compilers as well. However we don't want to cripplestdenv.xgcc
by taking away its ability to build dynamic libraries.It turns out that the only thing gcc needs the targetPlatform libc for is to emit a DT_NEEDED for
-lc
intolibgcc.so
. That's it! And since we don't usegccCrossStageStatic
to build anything other than libc, it's safe to omit theDT_NEEDED
because thatlibgcc
will never be loaded by anything other thanlibc
. Solibc
will already be in the process's address space.Other people have noticed this; crosstool-ng has been using this approach for a very long time:
https://github.com/crosstool-ng/crosstool-ng/blob/36ad0b17a732aaffe4701d5d8d410d6e3e3abba9/scripts/build/cc/gcc.sh#L638-L640
Closes #240484
Closes #240435
Things done
gcc6
(cross from x86_64-linux)pkgsCross.aarch64-multiplatform
.pkgsBuildTarget.gcc48
.pkgsBuildTarget.gcc49
.pkgsBuildTarget.gcc6
.pkgsBuildTarget.gcc7
.pkgsBuildTarget.gcc8
.pkgsBuildTarget.gcc9
.pkgsBuildTarget.gcc10
.pkgsBuildTarget.gcc11
.pkgsBuildTarget.gcc12
.pkgsBuildTarget.gcc13