cross: regression with armv7 pkgsStatic (error: C compiler cannot create executables)

[samueldr]

Describe the bug
With #101606 many pkgsCross.armv7l-hf-multiplatform.pkgsStatic. attributes that were working stopped building.

To Reproduce

 $ env -i nix-build -A pkgsCross.armv7l-hf-multiplatform.pkgsStatic.hello
[...]
configure: error: in `/build/hello-2.10':                                                                                                    
configure: error: C compiler cannot create executables                                                                                       
See `config.log' for more details                                                                                                            
builder for '/nix/store/ik89nsfcnjwrb5vz7chwc2accp604grh-hello-2.10-armv7l-unknown-linux-musleabihf.drv' failed with exit code 77            

Additional context
Reverting 4e9dc46 makes many thing work again.

I'm not sure what should be done here. On one hand I assume PIE is desired. But it definitely provably breaks things with cross-compilation.

It seems to be limited to armv7; the equivalent aarch64 builds work fine.

Notify maintainers
@Ericson2314 @matthewbauer as cross-compilation aficionados
@utsl42 author of the change

[r-burns]

Relevant config.log excerpt:

configure:4014: checking whether the C compiler works
configure:4036: armv7l-unknown-linux-musleabihf-gcc    conftest.c  >&5
/nix/store/lxzkkscp52ighr658k33wl71749rf8sk-armv7l-unknown-linux-musleabihf-binutils-2.35.1/bin/armv7l-unknown-linux-musleabihf-ld: /nix/store/8939rxc7d16hh25cxbcgym0pk9202s2a-armv7l-unknown-linux-musleabihf-stage-final-gcc-debug-10.2.0/lib/gcc/armv7l-unknown-linux-musleabihf/10.2.0/crtbeginT.o: relocation R_ARM_MOVW_ABS_NC against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/nix/store/8939rxc7d16hh25cxbcgym0pk9202s2a-armv7l-unknown-linux-musleabihf-stage-final-gcc-debug-10.2.0/lib/gcc/armv7l-unknown-linux-musleabihf/10.2.0/crtbeginT.o:(.init_array+0x0): dangerous relocation: unsupported relocation
/nix/store/8939rxc7d16hh25cxbcgym0pk9202s2a-armv7l-unknown-linux-musleabihf-stage-final-gcc-debug-10.2.0/lib/gcc/armv7l-unknown-linux-musleabihf/10.2.0/crtbeginT.o:(.fini_array+0x0): dangerous relocation: unsupported relocation
collect2: error: ld returned 1 exit status
configure:4040: $? = 1
configure:4078: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "GNU Hello"
| #define PACKAGE_TARNAME "hello"
| #define PACKAGE_VERSION "2.10"
| #define PACKAGE_STRING "GNU Hello 2.10"
| #define PACKAGE_BUGREPORT "[email protected]"
| #define PACKAGE_URL "http://www.gnu.org/software/hello/"
| #define PACKAGE "hello"
| #define VERSION "2.10"
| /* end confdefs.h.  */
| 
| int
| main ()
| {
| 
|   ;
|   return 0;
| }
configure:4083: error: in `/build/hello-2.10':
configure:4085: error: C compiler cannot create executables

[veprbl]

We could do something like

gcc.hardeningUnsupportedFlags = optional (stdenv.hostPlatform.isAarch32 && stdenv.hostPlatform.isStatic) "pie";

[samueldr]

@veprbl I'm not sure where hardeningUnsupportedFlags are defined. I'm not sure where it should be defined in an appropriate manner for all compilers(?). The only current definition is in pkgs/development/compilers/gcc/4.8/default.nix. And it does not look like it would be automatically merged if it were to be inherited from a more global location.

[veprbl]

@samueldr I agree with your assessment. We would need to apply it to all needed compilers individually.

[samueldr]

I don't feel this is the right solution. (I could be wrong.)

Couldn't we work this into the conditional here instead?

https://github.com/NixOS/nixpkgs/pull/101606/files#diff-282a02cc3871874f16401347d8fadc90d59d7ab11f6a99eaa5173c3867e1a160R279

After all, this is where it is being set globally.

[veprbl]

@samueldr To my taste, the hardening flag logic doesn't belong to stdenv, but to cc/cc-wrapper. Especially given that we don't do any eval-time validation there.

[samueldr]

Though it is not the way it's done currently. The flag logic is in stdenv. I'm not interested into having to refactor something I don't really understand in depth, to get things back to a working state.

nixpkgs/pkgs/stdenv/generic/make-derivation.nix

Line 108 in a6840c5

supportedHardeningFlags = [ "fortify" "stackprotector" "pie" "pic" "strictoverflow" "format" "relro" "bindnow" ];

I see that there's already a conditional for musl to add pie, why not fix it there? After all, it's there that it's being globally added.

nixpkgs/pkgs/stdenv/generic/make-derivation.nix

Lines 109 to 111 in a6840c5

	defaultHardeningFlags = if stdenv.hostPlatform.isMusl
	then supportedHardeningFlags
	else lib.remove "pie" supportedHardeningFlags;

[veprbl]

@samueldr Speaking of musl, shouldn't pkgsStatic set isMusl = true? Could that be the actual issue at hand?

[samueldr]

It does already:

nix-repl> pkgsStatic.hostPlatform.isMusl
true

And the conditional removes "pie" when it's not musl, so if I understand things right, it'd be rewritten

stdenv.hostPlatform.isMusl && !(stdenv.hostPlatform.isAarch32 && stdenv.hostPlatform.isStatic)

Though, non-static armv7 with musl should be verified too.

[veprbl]

@samueldr I got the condition wrong. This looks like a relevant bug
https://bugs.launchpad.net/ubuntu/+source/gcc-4.4/+bug/503448
Your proposed solution appears reasonable to me.

[samueldr]

Things may be more subtle than expected.

I hadn't tested in depth with aarch64, only compiled to see if it was compiling still. (I'm working on tests to check things)

pkgsCross.aarch64-multiplatform.pkgsStatic.hello when "pie" is in the hardening flags, produces dynamic binaries! Furthermore, it segfaults!

Still investigating, will open a PR with the changes that at least works around the regression, and maybe will be the solution.

EDIT: pkgsStatic.hello also produces a dynamically linked binary that segfaults on aarch64-linux (native compilation). Note that pkgsStatic.hello on x86_64-linux is just fine.