Create a Cross Compiler for a Bare Bones Freestanding Program (Bootloader/OS)

[CMCDragonkai]

Issue description

Hey @Ericson2314 I've been following your work on lib.systems, and I looked up the http://wiki.osdev.org/GCC_Cross-Compiler tutorial, and I found that the basic target of i686-elf or i686-pc-none-elf doesn't seem supported by NixOS at the moment.

The error message is simply: error: Unknown ABI: elf.

The main issue here is that lib.systems.elaborate cannot parse the bare bones target. And from looking at the nix expressions for gcc and binutils, it appears that bare bones target hasn't been considered.

What are your thoughts on this?

[Ericson2314]

Ah that osdev tutorial. Is elf really an ABI? Or is it just something just something thrown in the name? It's a bit confusing that sometimes those names are actually parsed, and sometimes they're just black-box names. Regardless, elf can be added in lib.systems.elaborate if we want.

stdenvNoCC along with gccCrossStageStatic can get you a wrapped toolchain without libc (CC @cleverca22), but I'd like to see something better as per #26004

[Ericson2314]

Also, not sure if you saw the comment in lib/systems/parse.nix, but I try to make the parsing follow http://llvm.org/doxygen/Triple_8cpp_source.html somewhat. I guess there is an elf there, interesting.

[CMCDragonkai]

@Ericson2314 I'm not sure if elf is really an ABI. But since you're using the LLVM parsing system, I want to point out that the equivalent target specification for i686-elf on gcc for llvm is in fact i686-pc-none-elf. Where did I get this? Here:

--target=i686-pc-none-elf -march=i686
http://wiki.osdev.org/LLVM_Cross-Compiler#Usage

So if LLVM supports such a target flag, then perhaps /lib/systems/parse.nix should as well?

I did have a skim over the llvm source, but I'm not familiar with LLVM's codebase, so I don't have a comment there. But where did you mean when you said "I guess there is an elf there, interesting."?

The other issue is that there is no --target configure flag set for binutils expression atm at master. While the osdev tutorial says that there needs a target flag setting. Also you even have a comment about this: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/tools/misc/binutils/default.nix#L93-L99 I'm not sure if this is relevant as well.

One more question. How does crossSystem translate to buildPlatform, hostPlatform and targetPlatform? Because I'm running on x86_64-unknown-linux-gnu. And so I expect that buildPlatform and hostPlatform to be both x86_64-unknown-linux-gnu, because I am building the compiler and running the compiler on the same system, but I'm targeting code to a freestanding i686-elf. I did have a read of the new manual regarding these three properties, but I didn't quite understand this particular situation.

[CMCDragonkai]

@Ericson2314 is there some documentation regarding the usage of stdenvNoCC and gccCrossStageStatic?

[Ericson2314]

But where did you mean when you said "I guess there is an elf there, interesting."?

http://llvm.org/doxygen/classllvm_1_1Triple.html#a83e907e55fa50e093caa96a0aff96201a456b64e26b8bcdbd8294689615d8a055 is the enum variant or whatever, parseFormat is the function in the source I linked originally. The use of this startsWith and endsWith implies to me that LLVM would recognize a name like i686-pc-linux-gnu-elf, and so i686-pc-none-elf is actually skipping the 4th component.

[Ericson2314]

--target configure flag set for binutils

https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/tools/misc/binutils/default.nix#L94 means we do pass it when it differs from --host. That comment is saying I'd like to always pass it, but that means always having prefixed binaries---a good idea but a big breaking change I rather save for 18.03.

[Ericson2314]

How does crossSystem translate to buildPlatform, hostPlatform and targetPlatform?

See https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/cross/default.nix. This adds two additional bootstrapping stages, the first of which changes just the target platform, and the second of which also changes the host platform.

If you have any suggestions on how I could make the docs clearer, I'd love to hear them!

[CMCDragonkai]

If you have any suggestions on how I could make the docs clearer, I'd love to hear them!

Examples for common use cases would be good. Kind of like. If I want to build on A, run on B, targeting C, how would I write a shell.nix/default.nix to do such a thing. Cross system specifications basically. Currently I only learned about the format of crossSystem when looking into lib/systems/default.nix.

I would like the simple specification of i686-pc-none-elf to work. Is the main issue right now just the parsing of this specification? If I were to look into fixing this, would there be other incompatibilities such as the current way gcc/binutils.. etc are specified that would prevent me from creating a freestanding executable?

See https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/cross/default.nix. This adds two additional bootstrapping stages, the first of which changes just the target platform, and the second of which also changes the host platform.

I see, so this means basically either I build and run on the same A, while targeting B, or I build on A, while running on B targeting B. Is there no use case for building on A, running on B and targeting C?

In the mean time, I've been trying to just build the cross compilers directly using nix-build, and while cross binutils worked great, cross gcc failed with the same errors occurring here: #27889 This happened with gcc 5 and gcc 4.9.

[Ericson2314]

Examples for common use cases would be good. Kind of like. If I want to build on A, run on B, targeting C, how would I write a shell.nix/default.nix to do such a thing.

Ok, I'll keep that in mind.

Cross system specifications basically. Currently I only learned about the format of crossSystem when looking into lib/systems/default.nix.

Did you see lib/systems/examples.nix? I think that's in the manual, but it could be emphasized more in conjunction with the above.

Is the main issue right now just the parsing of this specification?

Well, the parser needs to understand it, not merely accept it. Based on the LLVM, understanding would be adding a new field in thinggy.parsed.

[Ericson2314]

Is there no use case for building on A, running on B and targeting C?

Correct. It would be cool if individual packages worked in that case, but I don't see why such a bootstrapping stage would be necessary. It might seem like it would save work, but as it stands, compilers like gcc are built with their own runtime for emitted programs, so a A->B->C gcc would require an A->A->C gcc anyways. What's' the point of B when one can just use that?

[Ericson2314]

cross binutils worked great, cross gcc failed

This is A->A->B, not A->B->B? If we need to do it for any set of platforms, we should probably just unset CPP across the board for gcc.

[CMCDragonkai]

I did finally build the gcc cross compiler but outside of the nixpkgs infrastructure. This required using sources of its dependencies gmp mpfr libmpc that was brought in from the the ./contrib/download_prerequisites script inside gcc source. I was hoping to be able to just use the nixpkgs provided gmp, mpfr, and libmpc, however it appears I was missing some flags that is meant to go to the configure script, either that or I don't understand nix-build vs nix-shell well enough (for example, what would $out be used inside nix-shell!?) My experiments are happening here: https://github.com/MatrixAI/Kernel-Experiments

This was A->A->B. I kind of see what you mean by A->B->C being not necessary, but hopefully that can also be made clear on the manual as well.

Well, the parser needs to understand it, not merely accept it. Based on the LLVM, understanding would be adding a new field in thinggy.parsed.

What is thingy.parsed? Is that a field in the resulting attribute set after elaborate? I also meant if there would be potential pitfalls in the expressions for gcc and binutils that would prevent i686-pc-none-elf from working.

[Ericson2314]

sorry by "thingy" I meant any of buildPlatform hostPlatform targetPlatform localSystem or crossSystem, as they all share the same structure. Indeed that would come from the parsing call in elaborate.

I can't predict what autoconf with gcc and binutils would do with that config---but evidently something interesting based on the build failures you ran into. Then again, it could be cross compilation will generally be broken until #26805 is merged.

[jparris]

#26805 has been merged the documentation is very thin and seems to focus on nix-build. Can some one on help me understand how to create a default.nix which results in a nix-shell that contains i686-elf-{gcc,ldd,...}? Cheers, Jon

[Ericson2314]

#48286 might help you

[matthewbauer]

Yeah 52dbd33 should fix this. Let me know if there are issues using it.

[matthewbauer]

We don't precompile the i686-pc-elf target yet though. You can add it by adding it to examples.nix.