[Backport release-23.05] gccCrossStageStatic: enable dynamic libraries, rename it #261076
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 6 commits
September 24, 2023 15:50
This commit adds `hasSharedLibraries` to `lib.systems`. We need `plat.hasSharedLibraries` in order to know whether or not to expect `gcc` (and many other tools) to emit shared libraries (like `libgcc_s.so`). Many of the GNU build scripts are smart enough that if you configure them with `--enable-shared` on a platform (such as `arm-none-eabi`) that doesn't support dynamic linking, they will simply skip the shared libraries instead of aborting the `configurePhase`. Unfortunately the missing shared libraries in the final build product cause very hard-to-troubleshoot problems later on. The alternative to introducing `hasSharedLibraries` would be to set `isStatic` in these situations. However doing so causes `make-derivation.nix` to insert `-static` between the `pname` and `hostPlatform` suffix, which is undesirable. If at some point in the future we eliminate the `-static` suffix, then `hasSharedLibraries` can be made equal to `!isStatic`. (cherry picked from commit 6980e6b)
(cherry picked from commit e41f217)
…rgetLibc
This commit allows `gccCrossStageStatic` to build dynamically-linked
libraries. Since is no longer restricted to building static
libraries its name is no longer appropriate, and this commit also
renames it to the more-accurate `gccWithoutTargetLibc`.
By default, you can't build a gcc that knows how to create dynamic
libraries unless you have already built the targetPlatform libc.
Because of this, our gcc cross-compiler is built in two stages:
1. Build a cross-compiler (gccCrossStageStatic) that can build
only static libraries.
2. Use gccCrossStageStatic to compile the targetPlatform libc.
3. Use the targetPlatform libc to build a fully-capable cross
compiler.
You might notice that this pattern looks very similar to what we do
with `xgcc` in the stdenv bootstrap. Indeed it is! I would like to
work towards getting the existing stdenv bootstrap to handle cross
compilers as well. However we don't want to cripple `stdenv.xgcc`
by taking away its ability to build dynamic libraries.
It turns out that the only thing gcc needs the targetPlatform libc
for is to emit a DT_NEEDED for `-lc` into `libgcc.so`. That's it!
And since we don't use `gccCrossStageStatic` to build anything other
than libc, it's safe to omit the `DT_NEEDED` because that `libgcc`
will never be loaded by anything other than `libc`. So `libc` will
already be in the process's address space.
Other people have noticed this; crosstool-ng has been using this
approach for a very long time:
https://github.com/crosstool-ng/crosstool-ng/blob/36ad0b17a732aaffe4701d5d8d410d6e3e3abba9/scripts/build/cc/gcc.sh#L638-L640
(cherry picked from commit 2affd45)
This commit renames the `crossStageStatic` argument to the `gcc` expression to `withoutTargetLibc`. See previous commit for details. (cherry picked from commit 443dfc4)
We want a `libgcc_s.so` to be built by the first stage
cross-compiler (withoutTargetLibc), since that is the compiler which
will compile the target libc.
This commit accomplishes that, by making three changes:
1. Replacing the `targetPlatform.libc == "msvcrt" &&` conditional
with `enableShared`, so that the code which cross-build
`libgcc_s.so` is used for all cross compilers capable of emitting
shared libraries.
2. Removing the `targetPlatform == hostPlatform` guard from the code
which produces the `libgcc` output.
3. Looking for build products in in "lib/${targetPlatform.config}/"
rather than "lib/", so we will find them when cross compiling.
(cherry picked from commit 63305d0)
This commit is reverted in NixOS#240596 (which must go to staging). (cherry-picked from commit 96a2f1b)
avnik
requested review from
matthewbauer,
alyssais and
Ericson2314
as code owners
|
Well, I was too optimistic, at least mingw cross for gcc12 is broken |
|
Yeah, I kinda gave up on backporting it. This release cycle had an enormous amount of work done on cross compilation... there's just too much drift at this point unfortunately. |
|
Closing since NixOS 23.05 is EOL. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of changes
Summary (from #238154)
This PR allows gccCrossStageStatic to build dynamically-linked libraries. Since is no longer restricted to building static libraries its name is no longer appropriate, and this PR also renames it to the more-accurate gccWithoutTargetLibc.
@amjoseph-nixpkgs @mikatammi
Things done
-Built to test gcc versions
pkgsCross.aarch64-multiplatform
In addition I tried to cross-build https://github.com/tiiuae/ghaf project, using this backport as base nixpkgs branch. It builds and boots on my jetson oring AGX (~1100 userspace packages)
I keep intact "kludge to avoid mass-rebuild", may be worth to drop it, and switch this PR to staging-23.05