including syslimits.h fails if glibc is in build inputs

[lucabrunox]

Failure: http://hydra.nixos.org/build/23953373/nixlog/1/raw

In file included from /tmp/nix-build-virtualbox-5.0.0-3.18.19.drv-0/VirtualBox-5.0.0/src/VBox/ExtPacks/VBoxDTrace/include/VBoxDTraceLibCWrappers.h:33:0,
                 from /tmp/nix-build-virtualbox-5.0.0-3.18.19.drv-0/VirtualBox-5.0.0/src/VBox/ExtPacks/VBoxDTrace/onnv/cmd/dtrace/dtrace.c:71:
/nix/store/84ff6922izmz0g6akrpslibvabf7xf40-gcc-4.9.3/lib/gcc/i686-pc-linux-gnu/4.9.3/include-fixed/syslimits.h:7:25: error: no include path in which to search for limits.h
 #include_next <limits.h>

It seems to be a toolchain issue on i686 only for virtualbox:

a.c:

#include <limits.h>
#include <syslimits.h>

void main() {
}

Test: gcc a.c

OK x64: nix-shell -A linuxPackages.virtualbox

OK x86 stdenv: nix-shell -A pkgsi686Linux.utillinux

FAIL x86 for virtualbox: nix-shell -A pkgsi686Linux.linuxPackages.virtualbox

cc @vcunat @aszlig

[lucabrunox]

It's a problem with NIX_CFLAGS_COMPILE, unsetting them in the nix-shell for virtualbox fixed the problem.

This one is the problem: -isystem /nix/store/rrn35v5334h8izc9g5qvzh81lf12f402-glibc-2.21/include . It's the pkgsi686Linux.glibc .

Now it's weird.

OK x64: nix-shell -A linuxPackages.virtualbox . It has glibc include in the cflags.

FAIL x64: nix-shell -A utillinux . I added the same glibc include in the cflags.

FAIL x86: nix-shell -A pkgsi686Linux.linuxPackages.virtualbox .

FAIL x86: nix-shell -A pkgsi686Linux.utillinux . I added the same glibc include in the cflags.

So it seems that the x64 virtualbox build is lucky

[vcunat]

Note that these don't use standard stdenv on 64-bit but the multilib one.

[aszlig]

bisecting...

[aszlig]

Done. The issue seems to be in commit e08b9ab.

[aszlig]

Indeed, after reverting e08b9ab in master, VirtualBox compiles fine on i686-linux and x86_64-linux, now the interesting part is: why? Because that commit seems to be architecture-unrelated.

[lucabrunox]

@aszlig I think because previously we were linking against libiconv. Now instead libiconv is glibc and it gets to NIX_CFLAGS_COMPILE and breaks the syslimits.h. After the revert, can you check echo $NIX_CFLAGS_COMPILE|grep glibc in the virtualbox nix-shell? It should be empty.

[vcunat]

Ah, of course, I think the problem should be that it's a different glibc than in the stdenv it uses, because it uses a multi-lib stdenv.

[vcunat]

Actually, no, that doesn't make sense, because problems occur only on 32-bit and the stdenv there is equal to the usual 32-bit one (no real multi-lib).

[lucabrunox]

@vcunat if you read carefully what I (tried) explained above, that error also happens on x64 when glibc is in NIX_CFLAGS_COMPILE. But, virtualbox luckily compiles on x64 for some other reason.

[vcunat]

OK, filtering out that glibc include path fixes the build, so I pushed that for now.

It seems some interaction with usage of #include_next, but I don't see the particular cause nor why it doesn't appear everywhere else. Perhaps having libiconv = glibc; on linux isn't a good thing.

[lucabrunox]

Possible problem:

• Without glibc in cflags: include limits.h is found in gcc include-fixed.
• With glibc in cflags, limits.h is first found in glibc and then syslimits.h from gcc breaks.

[lucabrunox]

This happens also on other systems, e.g. ubuntu when adding -isystem /usr/include. In other words, the correct fix is that glibc should not be in -isystem because limits.h must be found in gcc not in glibc.

[lucabrunox]

And I wonder, why do we use -isystem instead of -I?

[lucabrunox]

Relevant: 6671bb8

[lucabrunox]

So @edolstra you appear to have some insight about that -isystem. Why isn't -I enough?

[lucabrunox]

Possible solution: use -idirafter, which is after -I and after system dirs. While -isystem is after -I but before system dirs which breaks gcc includes.

[lucabrunox]

Ok so this is what happens, with the help of @vcunat on IRC that let me understand what's going on:

By default, we have -idirafter glibc -idirafter gcc, which leads to the following default search path order for include_next: gcc, glibc.

Normal case: gcc, glibc. So syslimits.h is found in gcc, include_next limits.h is found in glibc. OK.

With -isystem glibc: glibc, gcc. As expected, because -isystem glibc comes before -idirafter gcc. So syslimits.h is found in gcc, but there's no limits.h next. KO.

With -I glibc: gcc, glibc. Unexpected, that's because -I ignores the option if it's a system directory, hence glibc is not moved! OK.

With -idirafter glibc: gcc, glibc. Not much expected, it's probably getting ignored because there's alread -idirafter glibc by default. OK.

What to do? :)

[vcunat]

Some -isystem discussion: 51713fb.

[vcunat]

This shouldn't be a blocker now, although the -isystem situation is (still) perhaps suboptimal.

[wizeman]

Maybe I'm mistaken, but gcc 4.9 seems to be failing due to what looks like a similar glibc/gcc header issue: https://hydra.nixos.org/build/24001011

[vcunat]

Yes, maybe. I know this problem; notice that stdenv.cc is gcc-4.9. I consider it a bootstrapping bug that our stdenv.cc != gcc (== gcc49).

[edolstra]

This can be fixed by adding gcc49 = cc to overrides in stdenvLinux in pkgs/stdenv/linux/default.nix, but we'll have to remember to keep that up to date when switching to a newer gcc.

[domenkozar]

@edolstra a friendly ping so you remember about this for gcc5

[vcunat]

In master we already have stdenv.cc == gcc. I suppose it was fixed with #2817.

[Mathnerd314]

Right, now gcc != gcc49. But nothing uses gcc49 so it's not a problem.

Perhaps this issue could be fixed by setting libiconv = stdenv? Since (g)libc is included in stdenv, and stdenv acts as a no-op in buildInputs.

[fpletz]

Works as of 16.09.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/where-is-limits-h/50535/2