PublishAot should have better error messages when cross-compiling

[richlander]

I tried to follow the excellent guidance at https://devblogs.microsoft.com/dotnet/improving-multiplatform-container-support/. I knew it wouldn't work, but wanted to see the error case. We seem to be relying on the underlying toolchain for the error case here. It's a bit rough given the guidance for the other scenarios. I'd like to see a better error message.

7.098   Generating native code
9.915   ILC: Method '[releasejsonapp]Program.<Main>$(string[])' will always throw because: Failed to load assembly 'Microsoft.AspNetCore'
13.60   /usr/bin/ld.bfd: unrecognised emulation mode: elf_x86_64
13.61   Supported emulations: aarch64linux aarch64elf aarch64elf32 aarch64elf32b aarch64elfb armelf armelfb aarch64linuxb aarch64linux32 aarch64linux32b armelfb_linux_eabi armelf_linux_eabi
13.63 clang-16 : error : linker command failed with exit code 1 (use -v to see invocation) [/source/releasejsonapp.csproj]
13.64 /root/.nuget/packages/microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/build/Microsoft.NETCore.Native.targets(364,5): error MSB3073: The command ""clang" "obj/Release/net8.0/linux-musl-x64/native/releasejsonapp.o" -o "bin/Release/net8.0/linux-musl-x64/native/releasejsonapp" -fuse-ld=bfd /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/sdk/libbootstrapper.a /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/sdk/libRuntime.ServerGC.a /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/sdk/libeventpipe-enabled.a /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/sdk/libstdc++compat.a /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/framework/libSystem.Native.a /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/framework/libSystem.IO.Compression.Native.a /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/framework/libSystem.Net.Security.Native.a /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/framework/libSystem.Security.Cryptography.Native.OpenSsl.a --target=x86_64-alpine-linux-musl -g '-Wl,-rpath,$ORIGIN' -Wl,--build-id=sha1 -Wl,--as-needed -pthread -ldl -lz -lrt -lm -pie -Wl,-pie -Wl,-z,relro -Wl,-z,now -Wl,--eh-frame-hdr -Wl,--discard-all -Wl,--gc-sections" exited with code 1. [/source/releasejsonapp.csproj]

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

I tried to follow the excellent guidance at https://devblogs.microsoft.com/dotnet/improving-multiplatform-container-support/. I knew it wouldn't work, but wanted to see the error case. We seem to be relying on the underlying toolchain for the error case here. It's a bit rough given the guidance for the other scenarios. I'd like to see a better error message.

7.098   Generating native code
9.915   ILC: Method '[releasejsonapp]Program.<Main>$(string[])' will always throw because: Failed to load assembly 'Microsoft.AspNetCore'
13.60   /usr/bin/ld.bfd: unrecognised emulation mode: elf_x86_64
13.61   Supported emulations: aarch64linux aarch64elf aarch64elf32 aarch64elf32b aarch64elfb armelf armelfb aarch64linuxb aarch64linux32 aarch64linux32b armelfb_linux_eabi armelf_linux_eabi
13.63 clang-16 : error : linker command failed with exit code 1 (use -v to see invocation) [/source/releasejsonapp.csproj]
13.64 /root/.nuget/packages/microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/build/Microsoft.NETCore.Native.targets(364,5): error MSB3073: The command ""clang" "obj/Release/net8.0/linux-musl-x64/native/releasejsonapp.o" -o "bin/Release/net8.0/linux-musl-x64/native/releasejsonapp" -fuse-ld=bfd /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/sdk/libbootstrapper.a /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/sdk/libRuntime.ServerGC.a /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/sdk/libeventpipe-enabled.a /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/sdk/libstdc++compat.a /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/framework/libSystem.Native.a /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/framework/libSystem.IO.Compression.Native.a /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/framework/libSystem.Net.Security.Native.a /root/.nuget/packages/runtime.linux-musl-x64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23329.7/framework/libSystem.Security.Cryptography.Native.OpenSsl.a --target=x86_64-alpine-linux-musl -g '-Wl,-rpath,$ORIGIN' -Wl,--build-id=sha1 -Wl,--as-needed -pthread -ldl -lz -lrt -lm -pie -Wl,-pie -Wl,-z,relro -Wl,-z,now -Wl,--eh-frame-hdr -Wl,--discard-all -Wl,--gc-sections" exited with code 1. [/source/releasejsonapp.csproj]

Author:	richlander
Assignees:	-
Labels:	untriaged, area-NativeAOT-coreclr
Milestone:	-

[jkotas]

Can we do a sanity check that there is a rootfs setup when somebody tries to do linux-x64<->linux-arm64 cross-compilation and error out if there is none?

Somewhere around here:

runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Publish.targets

Lines 71 to 72 in 7980421

	<Error Condition="'$(IlcHostPackagePath)' == '' and '$(RuntimePackagePath)' != '' and ('$(_hostArchitecture)' == 'x64' or '$(_hostArchitecture)' == 'arm64')"
	Text="Add a PackageReference for '$(_hostPackageName)' to allow cross-compilation for $(_targetArchitecture)" />

[jkotas]

Can we do a sanity check that there is a rootfs setup when somebody tries to do linux-x64<->linux-arm64 cross-compilation

@agocke pointed out that it is valid to cross-compile without rootfs. The rootfs check would be too restrictive.

I am not sure whether we can provide a better actionable error message for incorrectly configured native toolchains.

[MichalStrehovsky]

Can we do a sanity check that there is a rootfs setup when somebody tries to do linux-x64<->linux-arm64 cross-compilation

@agocke pointed out that it is valid to cross-compile without rootfs. The rootfs check would be too restrictive.

I am not sure whether we can provide a better actionable error message for incorrectly configured native toolchains.

Yep, it would break https://www.nuget.org/packages/PublishAotCross, for example.

Feels like the best we could do is to parse the output of the linker looking for "known bad" strings and translate them to something higher level. But it's super fragile, only works for english locales, and I would resort to it only if many people start hitting this. It feels like we could start by documenting the "known bad" string in the general crosscompilation doc as a sort of "troubleshooting guide".

[richlander]

Sounds good.