Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for FreeBSD #14537

Open
ghuntley opened this issue May 4, 2015 · 806 comments · Fixed by #58085
Open

Support for FreeBSD #14537

ghuntley opened this issue May 4, 2015 · 806 comments · Fixed by #58085
Labels
area-Meta enhancement Product code improvement that does NOT require public API changes/additions needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration os-freebsd FreeBSD OS
Milestone

Comments

@ghuntley
Copy link
Member

ghuntley commented May 4, 2015

Updated proposal from 2017/9

Proposal (by @karelz - https://github.com/dotnet/corefx/issues/1626#issuecomment-329840518) will be updated in top-post based on further discussion and proposal changes.

We discussed community-driven port for FreeBSD with @RussellHaley (from FreeBSD community) and @wfurt (from .NET Core team) who both expressed interest in the work.
Here's a plan proposal we put together (feedback / suggestions are welcome):

  1. Produce binaries in CoreCLR & CoreFX repo targeting FreeBSD - using hacks is fine
    • Hard to parallelize, @wfurt will work on that
    • The build can be mix of builds from other platforms (Mac, Linux) targeting FreeBSD
    • We will need documented steps (on FreeBSD wiki) to reproduce the build with FreeBSD-specific bug fixes
  2. Run & stabilize CoreCLR tests (using corerun)
    • Tests may be built on another platform
    • Goal: Provides basic quality of runtime
  3. Run & stabilize CoreFX tests (using corerun)
    • Tests may be built on another platform
    • Note this requires xunit. We believe, based on our past porting experience, once [2] is done, xunit will just work.
    • This can be in theory parallelized with [2] - it may require shortcutting xunit (e.g. generate static execution recipe on another platform)
    • We can expose new OSPlatform API for FreeBSD when the pass rate is reasonable: see dotnet/corefx#23989
  4. Full stack build on FreeBSD (using corerun as bootstrapper from [1]-[3])
    • We will need all tools (nuget, msbuild, roslyn) to work on boostrapping .NET Core
  5. Installers (FreeBSD ports)
    • First-stage: Using product binaries from nuget feeds
    • Second-stage: Build product from source (blocked on build from source effort)
    • Requires FreeBSD community expertise and guidance on design
    • Note: We can link FreeBSD packages also from official .NET Core download pages as community-support packages
  6. Regular build and test runs on FreeBSD
    • Goal: Make sure changes in .NET Core repos breaking FreeBSD are known early
    • Design needed
    • Requires FreeBSD community expertise and guidance on design

Operation principles:

  • Changes in [2]-[4] should be done primarily in CoreCLR/CoreFX repos (due to CLA signing requirements, code reviews from .NET Core team experts/members. etc.)
  • We will track high-level work on this issue. Specific bugs will be filed as separate issues.

If anyone is interested in helping, please let us know here. We can easily distribute work items from [2] & [3] above once we are far enough with [1].


Original proposal from @ghuntley from 2015/5

This issue is to discuss unit(s) of work to actually produce FreeBSD assemblies for corefx.

@stephentoub - There's what's likely a more pressing issue, which is actually building for FreeBSD. Today, when we need to specialize an assembly for a particular platform, we effectively have three builds, producing three different managed assemblies: Windows, Linux, OSX. Sounds like at least for now we'll need a fourth, FreeBSD. I suggest you start by modifying the build to support an IsFreeBSD property (or just IsBSD of you think there's a high chance that the implementations across BSDs will be the same even with varied kernels) along with the appropriate OSGroup targets. That can then be used in the csproj files as needed to specialize an assembly with FreeBSD-specific code.

Related issue(s)

/cc: @janhenke @josteink

@josteink
Copy link
Member

josteink commented May 4, 2015

There seems to be agreement as far as https://github.com/dotnet/corefx/issues/1576 is concerned.

When we also have a decision on https://github.com/dotnet/corefx/issues/1625 we should be able to start shipping some code.

@ghuntley
Copy link
Member Author

Agreement on #14536 has been reached by the portteam, unless MSFT chooses otherwise it will be FreeBSD. Issue dotnet/corefx#1999 will potentially be the issue that introduces the definition into the public API.

@josteink
Copy link
Member

Agreement on #14536 has been reached by the portteam, unless MSFT chooses otherwise it will be FreeBSD

If I read that right, this means that when dotnet/corefx#1999 is merged, we can consider this MSFT approving of the new public API, and can therefore press forward on the remaining issues with regular pull-requests without need for MSFT approval.

If so, that sounds good to me.

@ghuntley
Copy link
Member Author

Next steps as per dotnet/corefx#1999 (comment) are:

  1. The "FreeBSD port team" continues their work to get a FreeBSD version of CoreFX produced (tracked by dotnet/corefx#1626).
  2. The port team brings up enough of the CoreFX and CoreCLR stack on FreeBSD such that we can start running the CoreFX unit tests on FreeBSD.
  3. The tests reach some minimal quality level. I don't know exactly what this looks like yet, but I expect it means something like a majority of the tests pass. Ideally we would not have a bunch of specific tests disabled for only FreeBSD (compared to Linux and OSX, we wouldn't want to hold FreeBSD to a higher standard than the other *NIX platforms we have there).
  4. Working with the FreeBSD port team, the CoreFX team gets the CoreFX tests added to our CI system running on FreeBSD.
  5. Discuss merging a PR based for issue [public api] System.Runtime.Environment - OSName("FreeBSD") or OSName("BSD") #14536, which adds the property.

@josteink
Copy link
Member

That sounds like a fully reasonable plan to me.

@janhenke
Copy link
Member

Okay, then let's start the work on getting corefx to work.

@josteink
Copy link
Member

First obstacle in building corefx on FreeBSD seems to be mono. The build-script insists version 4.1 is required. @ajensenwaud did some work on this on the Frankfurt-host, but I'm not sure how complete it is.

I'll queue a build for now and see what the output looks like.

Edit: The (mono) build crashes with the following kicker at the end:

Making all in mini
make[1]: "/usr/home/josteink/mono/mono/mini/Makefile" line 2906: warning: duplicate script for target "%.exe" ignored
make[1]: "/usr/home/josteink/mono/mono/mini/Makefile" line 2899: warning: using previous script for "%.exe" defined here
  CC       genmdesc-genmdesc.o
In file included from genmdesc.c:9:0:
mini.h:17:34: fatal error: ./mono/metadata/loader.h: Too many levels of symbolic links
 #include <mono/metadata/loader.h>
                                  ^
compilation terminated.
*** Error code 1

Stop.
make[1]: stopped in /usr/home/josteink/mono/mono/mini
*** Error code 1

Stop.

@stephentoub
Copy link
Member

First obstacle in building corefx on FreeBSD seems to be mono

FWIW, I personally don't think this is the first obstacle. There are two build related issues:

  1. Building assemblies that work correctly on FreeBSD
  2. Building those assemblies on FreeBSD

(1) is critical, and is I believe what this issue is meant to be about. (2) is very nice to have, but lack of it doesn't prevent the creation of a great system for running managed code on FreeBSD.

You're of course free to prioritize however you see fit, but my recommendation would be to focus on (1) rather than (2).

Note that we still have issues building corefx on Linux and building it on OSX, such that our CI system builds the assemblies for those platforms on Windows; it then shuttles the resulting assemblies over to the target platform to execute the tests.

@josteink
Copy link
Member

That's fair enough. I just assumed that it would be easier to get general FreeBSD platform support baked into corefx if we could actually build it ourselves on FreeBSD.

I'll make do with Windows-initiated building for now and attempt to ninja together a build-configuration.

@akoeplinger
Copy link
Member

@josteink btw. corefx should now build on Mono 4.0.1.44.

@josteink
Copy link
Member

@akoeplinger Nice. That leaves me some hope we can get it running on FreeBSD too :)

@ajensenwaud
Copy link
Contributor

Good points. However if we really want corefx to be part of the FreeBSD environment, we really need it to be able to compile from source to get it into the Ports system.

I did hear that Mono 4.0.1.44 fixes a lot of these issues but have not had time to play with it yet. I know the ports team are updating the port Makefile as well as we speak with a new patch.

On 12 Jun 2015, at 20:21, Stephen Toub [email protected] wrote:

First obstacle in building corefx on FreeBSD seems to be mono

FWIW, I personally don't think this is the first obstacle. There are two build related issues:

Building assemblies that work correctly on FreeBSD
Building those assemblies on FreeBSD
(1) is critical, and is I believe what this issue is meant to be about. (2) is very nice to have, but lack of it doesn't prevent the creation of a great system for running managed code on FreeBSD.

You're of course free to prioritize however you see fit, but my recommendation would be to focus on (1) rather than (2).

Note that we barely have corefx building-on-Linux and building-on-OSX, such that our CI system builds the assemblies for those platforms on Windows; it then shuttles the resulting assemblies over to the target platform to execute the tests.


Reply to this email directly or view it on GitHub.

@stephentoub
Copy link
Member

Yes, I'm in no way disagreeing... being able to build corefx on Linux, OSX, and FreeBSD is important. I'm simply suggesting that from a priority perspective it's more important to be able to actually run corefx on Linux, OSX, and FreeBSD. 😉 If both can be worked on in parallel, all the better.

@ghost
Copy link

ghost commented Jun 13, 2015

@ghuntley,
would be super 🆒 if we have a markdown task checklist outlining what what is remaining:

- [x] task 1
- [ ] task 2
- [ ] task 3

renders as:

  • task 1
  • task 2
  • task 3

This will probably encourage others to score those feats and FreeBSD support will land rather sooner than anticipated! 😎

@janhenke
Copy link
Member

To my knowledge the following pieces of work in CoreFX are required for FreeBSD support:

13 Assemblies do not compile on their own and need FreeBSD specific changes. Mostly the Interop pieces that already exist for Linux/OS X (order by the occurrence in the build output):

I will try to update that list based on PRs opened and merged.

@josteink
Copy link
Member

FYI: PR dotnet/corefx#2039 merged

@josteink
Copy link
Member

Just trying to be ahead of the curve here... How do we plan to implement System.IO.FileSystem.Watcher ?

Iirc FreeBSD has no inotify such as Linux and Windows does (which is also why there is no Dropbox last time I checked). Will this be a potential source of trouble coming our way? Or does anyone have an idea for how to work around this?

@janhenke
Copy link
Member

I suggest we stub that out for the moment and throw a PlatformNotSupportedException as Stephen Toub suggested in the other topic (dotnet/corefx#2021 (comment)). Then we have at least a complete set of assemblies and we can continue to work on that particular issue without blocking further steps.

Would you mind opening a separate issue for that?

@ghuntley
Copy link
Member Author

Let's move System.IO.FileSystem.Watcher discussions to dotnet/corefx#2046

@ghost
Copy link

ghost commented Jun 15, 2015

Guys is there any such blocker for System.Diagnostics.Process?

@ghuntley
Copy link
Member Author

@jasonwilliams200OK added FreeBSD to S.RT.I.RI early this morning which was merged but the FreeBSD tests within CheckPlatformTests had to be backed out until dotnet/buildtools is updated.

@ghuntley
Copy link
Member Author

@jasonwilliams200OK there were some discussions last night about System.Diagnostics.Process in gitter which have been formalized into https://github.com/dotnet/corefx/issues/2070

@ghost
Copy link

ghost commented Jun 16, 2015

@ghuntley, thanks. I actually read those messages. System.Diagnostics.Process is a tricky one. AFAIK, io.js team had similar challenges with FreeBSD process management. Mono team has probably nailed it, so lets hope if @akoeplinger and co. could enlighten us on this matter? :)

@josteink
Copy link
Member

System.IO.FileSystem.DriveInfo

As discussed in the gitter, For this one I tried looking into basic usage of getmntinfo:

#include <sys/param.h>
#include <sys/ucred.h>
#include <sys/mount.h>
#include <stdio.h>

int main() {
  struct statfs *mntbuf;
  int mntsize = getmntinfo(&mntbuf, MNT_NOWAIT);

  for( int i = 0; i < mntsize; i++ ) {
    printf("%s\n", mntbuf[i].f_mntonname);
  }
}

Running that sample yielded this output:

$ ./a.out
/
/dev
/tmp
/usr/home
/usr/ports
/usr/src
/var/crash
/var/log
/var/mail
/var/tmp
/dev/fd
/usr/compat/linux/proc
/proc
$

So it seems it does what we need. The question is, should we do any type of filtering on the results?

Looking at the "intent" of the DriveInfo object, coming from the Windows world of .NET it has often been to enumerate the available locations to store or retrieve files (C:, D:, etc). But when using Unix hierarchical file-systems, returning / would be adequate to cover those needs.

So what should we return? What would be useful? Should even consider it being useful or not?

The Linux-version just dumps everything, except things set to be ignored:

https://github.com/dotnet/corefx/blob/master/src/System.IO.FileSystem.DriveInfo/src/System/IO/DriveInfo.Linux.cs#L98-L99

I tried putting in the following filter, but it didn't really change anything in terms of output:

    if ((mntbuf[i].f_flags != MNT_IGNORE)) {
        printf("%s\n", mntbuf[i].f_mntonname);
    }

Any opinions?

@ghost
Copy link

ghost commented Jun 21, 2015

@josteink, great diggings! Based on https://github.com/dotnet/corefx/issues/815#issuecomment-113825960 and https://github.com/dotnet/corefx/issues/1729, I think we should collaborate with @sokket to come up with a solution with works across different Unices.

@sec
Copy link
Contributor

sec commented Mar 5, 2024

@rmszc81 First problem is just the way it is, you have to do it.

For the second part, you have two choices:

  • either create local directory, download those nuget's (either from port build output or from mine's or @Thefrank releases) and add that directory as nuget source. You can do that per project or .nuget/NuGet/NuGet.Config
  • second option is to use private nuget feed with those, as we don't have nothing "semi-official", for example I'm maintaining my own feed with outputs both for x64 and arm64 - check the info/link in here https://github.com/sec/dotnet-freebsd-nuget-feed
  • you can albo build dotnet from ports, it will generate Private.SourceBuilt.Artifacts which should also contain all the needed nuget's (or you can use the one that's bootstraping dotnet port build, just make fetch should get them, then extract that to local dir and reference in nuget.config).

@nkosi23
Copy link

nkosi23 commented Mar 5, 2024

@sec Interesting! Since the NuGet packages are generated when building the port, would it be a good idea to add this custom NuGet feed automatically as a post-install script of the package?

We could for example copy the relevant packages under Private.SourceBuilt.Artifacts to a location such as: /usr/local/dotnet/nuget and create the .nuget/NuGet/NuGet.Config file pointing to this directory

@sec
Copy link
Contributor

sec commented Mar 5, 2024

That's good idea, we could copy FreeBSD specific nuget's after build and add pkg-message informing user to reference this directory in nuget.config - @arrowd what do you think?

@akoeplinger
Copy link
Member

akoeplinger commented Mar 5, 2024

Normally these packages are in the packs folder in the sdk root when source-building .NET for Linux distros. I think that doesn't happen for non-source-build but you can put them there.

@arrowd
Copy link
Contributor

arrowd commented Mar 5, 2024

Sounds good. @sec can you make a patch for the port yourself? It should be too hard and you seem to already know what and where to put stuff.

@akoeplinger
Copy link
Member

akoeplinger commented Mar 5, 2024

@rmszc81
Copy link

rmszc81 commented Mar 5, 2024

Hello everyone!

My solution was to run make fetch (as suggested by @sec), then, I extracted the contents of the artifacts package into /var/cache/nuget and added this directory to my Nuget.config file configuration.

So far, so good, the project was built, published to /app/bin inside the jail but, now I have a problem with the encrypted SQLite database, aka sqlcipher that says that some .so files are missing. And well, they are, lol.

As this problem is not related to the .NET framework, I'll go after the solution in the appropriated place. Besides, as my app has support to Postgres as well, not having sqlcipher is not an issue at all.

Anyway, I'd like to say thanks again to @sec, @nkosi23, @akoeplinger and @arrowd for the support. I went through all the links and I found a lot of cool stuff.

See you guys around.
Best!

@rmszc81

@sec
Copy link
Contributor

sec commented Mar 5, 2024

As for SQLite, it's old known issue (ericsink/SQLitePCL.raw#176) - simplest you can do is to either create symbolic link e_sqlite3.so in /usr/local/lib to your SQLite version.

@rmszc81
Copy link

rmszc81 commented Mar 6, 2024

hello @sec,
in this case, it's the e_sqlcipher.so.

I'll have a look on this in another time. But thanks anyway, you helped a lot already ^^

@sec
Copy link
Contributor

sec commented Mar 14, 2024

Well... As building 8.0.3 was no problem... Then 9.preview-2 to build from VMR require 9.preview-2 (just few versions behind, using preview-1 doesn't work), I really don't get it...

@davidchisnall
Copy link

1st, it was necessary to set allow.mlock=1 to the jail configuration. otherwise, it wasn't possible even to run a dotnet new command. problem solved.

If I remember correctly, the mlock call is there to provide support for the asymmetric lock, which requires executing a full barrier on other threads. This is implemented on FreeBSD by updating a page mapping that must be guaranteed to trigger IPIs.

FreeBSD 13.3 and 14 both include a Linux-compatible membarrier, and so the fallback code should no longer be required. This should remove the need for mlock.

Unfortunately, even though this work was sponsored by the FreeBSD Foundation, it landed without an accompanying man page so you have to poke at the headers and the git history to know that it's there (@emaste, perhaps the Foundation could also sponsor the docs?).

Note that, unfortunately, you can't use the header to detect the features that are supported because the header provides a full set of the Linux #defines but the kernel implements only a subset. I believe the subset is sufficient for .NET.

@Thefrank
Copy link
Contributor

So as the SDK does not accept "General Support" via issues...

How is the installer built now that the Installer repo is gone for all versions of dotNET? The SDK has always come across as "magic" to me.

From the old installer repo:
The RID was patched in GenerateBundledVersions.targets and if crossbuilding a patch was made to GetRuntimeInformation.targets so that it would include the targeted OS items instead of the host OS items. The former is gone entirely (how are supported RIDs generated?) and the later was trimmed down to a few lines, removing the FreeBSD reference entirely. Right now, I am just looking at dotNET6 but I assume this is similar across all in-support versions.

@jkotas
Copy link
Member

jkotas commented Jun 13, 2024

Installer repo is gone for all versions of dotNET?

The branches for shipped version of .NET are still there and in use.

@Thefrank
Copy link
Contributor

ah! I see my mistake. I was tracking SDK branches until a new one would come up, then follow that: 6.0.1xx -> 6.0.2xx -> etc. I was looking for a 6.0.423 which does not exist. From the core repo I see it only lists a 6.0.1xx branch for the SDK and there is a 6.0.131 in the installer repo even though it is unlisted in core. I will give this a shot!

pointy hat squarely on me here.

@Foxtrek64
Copy link
Contributor

Can we update the OP with what's left of the to-do list on this? I'm definitely interested in contributing if I can, especially considering that I personally want to see BSD support. There was a post here which aggregated a to-do list, but it has everything completed.

@Thefrank
Copy link
Contributor

@Foxtrek64

A quick summary of os-freebsd FreeBSD OS

Documentation:
#31238

Implementation:
#3320

  • Major.Minor is a bit more important as FreeBSD ABI changes can impact dotnet

#26633

  • We use libinotify shim instead of native kqueue.
  • This would remove a dependency but I am not sure how much faster / better it would be as neither libinotify nor kqueue are designed for monitoring large number of files; this hits mediaplayers and backup tools built with dotnet.
  • Something "better" would need to come from the FreeBSD side.

#10519

  • Not sure on this one. I think this is more FreeBSD issue than dotnet issue? We don't have libthr2 on FreeBSD

#8544

  • This covers all non-windows builds

No issue but mentioned:

#14537 (comment)


Pending PRs:

#103187

  • Checks system uses a FreeBSD 13 based image so this was missed.

Others I missed?

@Thefrank
Copy link
Contributor

Thefrank commented Jun 29, 2024

WALL OF STUFF WARNING

dotNET9 preview 5 cross compiles from linux with two small caveats:

  • GetRuntimeInformation.targets patch needed to be reapplied to the new file location. Native builds do not need this. AFAIK native builds require no patching currently!
  • ILCompiler made by runtime still contains Linux ELFs despite being labeled for freebsd-x64

/p:BundleRuntimePacks=true seems to work. The packs path contains the runtime and aspnetcore for FreeBSD as mostly symlinks.
/p:BundleNativeAotCompiler=true is not in preview 5 and hopefully will operate like BundleRuntimePacks

frank@freebsd:~/sdktest $ uname -a
FreeBSD freebsd 14.1-RELEASE FreeBSD 14.1-RELEASE releng/14.1-n267679-10e31f0946d8 GENERIC amd64
---- Built using the CBL Mariner image with FreeBSD 13.2 crossrootfs

frank@freebsd:~/sdktest $ ./dotnet --info
.NET SDK:
 Version:           9.0.100-preview.5.24307.3
 Commit:            35b2c21ea6
 Workload version:  9.0.100-manifests.6407b7e4
 MSBuild version:   17.11.0-preview-24279-02+b963c24ef

Runtime Environment:
 OS Name:     FreeBSD
 OS Version:  14
 OS Platform: FreeBSD
 RID:         freebsd-x64
 Base Path:   /home/frank/sdktest/sdk/9.0.100-preview.5.24307.3/

.NET workloads installed:
Configured to use loose manifests when installing new manifests.
There are no installed workloads to display.

Host:
  Version:      9.0.0-preview.5.24306.7
  Architecture: x64
  Commit:       a5cc707d97

.NET SDKs installed:
  9.0.100-preview.5.24307.3 [/home/frank/sdktest/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 9.0.0-preview.5.24306.11 [/home/frank/sdktest/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 9.0.0-preview.5.24306.7 [/home/frank/sdktest/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

----

frank@freebsd:~/sdktest $ dotnet publish -r freebsd-x64 console/console.csproj /p:PublishAot=true
-sh: dotnet: not found
frank@freebsd:~/sdktest $ ./dotnet publish -r freebsd-x64 console/console.csproj /p:PublishAot=true
    /home/frank/sdktest/console/console.csproj : error NU1101: Unable to find package runtime.freebsd-x64.Microsoft.DotNet.ILCompiler. No packages exist with this id in source(s): nuget.org
    /home/frank/sdktest/console/console.csproj : error NU1101: Unable to find package runtime.freebsd-x64.Microsoft.DotNet.ILCompiler. No packages exist with this id in source(s): nuget.org

---

Restore failed with 2 error(s) in 3.6s
frank@freebsd:~/sdktest $ ./dotnet publish -r freebsd-x64 console/console.csproj /p:PublishReadyToRun=true
    /home/frank/sdktest/console/console.csproj : error NU1101: Unable to find package Microsoft.NETCore.App.Crossgen2.freebsd-x64. No packages exist with this id in source(s): nuget.org

Restore failed with 1 error(s) in 3.9s

neither of these work without external NuGet packages.
https://reviews.freebsd.org/D44561 might be the better solution if items like crossgen2 and ilc are not able to be added outside of source building

Crossgen2 NuGet correctly contains FreeBSD ELFs:

$file crossgen2 
crossgen2: ELF 64-bit LSB pie executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 13.2, FreeBSD-style, BuildID[sha1]=54a7f1c2a4752435c2cffd15eeb959f609966907, stripped

ILCompiler does not: it contains a mix of FreeBSD libraries and Linux ELFs

$ ./ * | xargs file
./:                                           directory
./ILCompiler.RyuJit.pdb:                      Microsoft Roslyn C# debugging symbols version 1.0
./libclrjit_unix_x64_x64.so:                  ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, for FreeBSD 13.2, BuildID[sha1]=66177aebc4ab51f16fe1e6a5faa90a7ade09b674, stripped
./libclrjit_win_x86_x64.so:                   ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, for FreeBSD 13.2, BuildID[sha1]=79ecdf1053497bde0393928dee1a727bc6b6b6a1, stripped
./libclrjit_universal_arm_x64.so:             ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, for FreeBSD 13.2, BuildID[sha1]=b15d888e793cca18b6dd42b3b672f7144fbe45ec, stripped
./libjitinterface_x64.so:                     ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, for FreeBSD 13.2, BuildID[sha1]=02a2d4a17bcbd0ff35f3caca9252853f95529a3c, stripped
./ILCompiler.TypeSystem.pdb:                  Microsoft Roslyn C# debugging symbols version 1.0
./ILCompiler.DependencyAnalysisFramework.pdb: Microsoft Roslyn C# debugging symbols version 1.0
./ILCompiler.Compiler.pdb:                    Microsoft Roslyn C# debugging symbols version 1.0
./libclrjit_universal_arm64_x64.so:           ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, for FreeBSD 13.2, BuildID[sha1]=0b476dc684291af72ab673b77dccbbf7f386cbf8, stripped
./libclrjit_win_x64_x64.so:                   ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, for FreeBSD 13.2, BuildID[sha1]=a58827fd1b7ac68612408fa5e13c7db9091938a2, stripped
./ilc.pdb:                                    Microsoft Roslyn C# debugging symbols version 1.0
./ilc:                                        ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=41cb9e347020ad19b6402190528b968b6850c46f, stripped

Other than Linux ELFs in FreeBSD packs everything is going smooth from the cross compile side for .net 9


EDIT: 4 day later update

Using the most recent net9 preview tag, TargetOS=linux only appears three places in the binlog that are after Evaluation. All three are from ILCompiler.cspoj : The first seems to come as a return from ResolveReadyToRunCompilers and the other two (_PrepareForReadyToRunCompilation) and (_CreateR2RImages) use it. Should this be blocked from

<NativeAotSupported Condition="'$(TargetOS)' == 'freebsd' and '$(CrossBuild)' == 'true'">false</NativeAotSupported>
or error out from
_targetPlatform != hostPlatform ||
!GetCrossgenComponentsPaths())
{
Log.LogError(Strings.ReadyToRunTargetNotSupportedError);
return false;

There is no "Property reassignment" note in the binlog when this happens.

AFAIK Cross-OS is not supported for R2R or AOT when targeting FreeBSD. The Crossgen2 project seems to honor this as it skips this part during packaging:

<Target Name="RunPublishedCrossgen" AfterTargets="PublishCrossgen"
Condition="'$(TargetOS)' == '$(HostOS)' and '$(TargetArchitecture)' == '$(BuildArchitecture)' and '$(CrossBuild)' != 'true'">
and we still get a FreeBSD ELF from out of it.

I will try and work over this a bit more in the coming days and open an actual issue for this instead of burying it in an edit of a long post.

@sec
Copy link
Contributor

sec commented Jul 11, 2024

@Thefrank did you cross compile from VMR or from the repo's (runtime, aspnetcore and sdk (as it now contains installer)) ?

@Thefrank
Copy link
Contributor

Thefrank commented Jul 11, 2024

@sec AFAIK you can not use VMR for cross compile, but I have never actually tried. I will give it a shot for the most recently published tags.
For net6, 7, and 8 I use installer tracking the .1xx branches. VMR does this too. For net9 I am using SDK because installer simply does not have any tags past preview 4.

Summary of wall and issue:
Cross compiles for versions that use ILCompiler (i.e. 8 and 9) are less useful due to #104497. You will be unable to use the ILCompiler to bootstrap or for AOT as ILCompiler contains incorrect ELFs. There are some possible solutions for this listed in the linked thread but for now the only options are: native VMR (non-portable), native build via each repo (portable), pkg install dotnet8 ("non-portable"*, missing ILCompiler, Crossgen2, and other NuGets until https://reviews.freebsd.org/D44561 lands). For most people the pkg install method should be more than enough. Those looking to bootstrap will have to wait :)

edit: There is a typo in the tag for this repo: https://github.com/dotnet/runtime/releases/tag/9.0.0-preview.6.24327.7 <- all other tags both here and in other repos contain a leading v: https://github.com/dotnet/aspnetcore/releases/tag/v9.0.0-preview.6.24328.4

@logiclrd
Copy link
Contributor

I installed .NET 8.0.6 using pkg on a 14.1-RELEASE system and it's working pretty well. But, core host stubs don't work:

You must install .NET to run this application.

App: /srv/DQD.RealTimeBackup/DQD.RealTimeBackup.Web
Architecture: x64
App host version: 8.0.6
.NET location: Not found

Learn more:
https://aka.ms/dotnet/app-launch-failed

Download the .NET runtime:
https://aka.ms/dotnet-core-applaunch?missing_runtime=true&arch=x64&rid=freebsd-x64&os=freebsd.14&apphost_version=8.0.6

The application can be run by explicitly launching dotnet DQD.RealTimeBackup.Web.dll.

I've tracked this down to the default install location, per pal.unix.cpp, being /usr/share/dotnet, but the actual install location used by the package being /usr/local/dotnet.

If /usr/local/dotnet is put into /etc/dotnet/install_location then the problem goes away.

@Thefrank
Copy link
Contributor

@logiclrd should be fixed in net9 #100731 and with this https://reviews.freebsd.org/D44560

@logiclrd
Copy link
Contributor

@Thefrank Thanks for the reply :-) I took a look at that PR and I don't think it'll fix the problem at the runtime level. The FreeBSD-specific logic it adds still looks in /usr/share/dotnet, and that's not where pkg install dotnet put it. But, it looks like D44560 will do what I suggested, putting the path into /etc/dotnet/install_location. :-)

I don't know how to read Differential (that looks like what it's called?). Does being in that list mean it's already a part of the codebase? It says Needs Review, that doesn't mean it's unmerged though, like, say, an open PR on GitHub? When does it take effect??

@Thefrank
Copy link
Contributor

@logiclrd That differential is has not been commited. If/when that happens, it will be closed and it will have a commit attached to it in the "Details" section.

If you are using your own ports base, you can make the change locally by downloading, ("Download Raw Diff" on the sidebar), applying (via git), and then rebuilding the port. Unless you already maintain a local ports tree, I suggest waiting and continuing to use the workaround when needed.

@logiclrd
Copy link
Contributor

@Thefrank Maybe .NET 9 installs to a different path than .NET 8...

https://github.com/dotnet/runtime/pull/100731/files#diff-b2bd8511e88327d8f146bc460e79182cc6826e2b90039dffc4f5f89c5467d12aR608

        recv->assign(_X("/usr/local/share/dotnet"));

but:

root@laliari:/ # uname -a
FreeBSD laliari.logiclrd.cx 14.1-RELEASE FreeBSD 14.1-RELEASE releng/14.1-n267679-10e31f0946d8 GENERIC amd64
root@laliari:/ # dotnet --version
8.0.106
root@laliari:/ # which dotnet
/usr/local/bin/dotnet
root@laliari:/ # ls /usr/local/share/dotnet
ls: /usr/local/share/dotnet: No such file or directory
root@laliari:/ # 

This was done with simply pkg install dotnet on 14.1-RELEASE.

@Thefrank
Copy link
Contributor

@logiclrd net8 will not contain that fix unless it is manually added via patch from the port maintainer and/or D44560. net9 as of preview 6 does contain the fix.

the pkg makes a symlink between the binary location and is why which finds it in /usr/local/bin/dotnet
https://github.com/freebsd/freebsd-ports/blob/f8aa1411e9a7aeb48bef86f02e0da44acdad82fb/lang/dotnet/Makefile#L107
It also uses ${PREFIX} which is typically ${LOCALBASE} and that is usually /usr/local (more info here)
I expect the install location in the port to change so that it reflects the changes in the PR.

There is currently no net9 pkg

@logiclrd
Copy link
Contributor

@Thefrank I see that it is a symlink, but it is a symlink to /usr/local/dotnet/dotnet -- still no share in the mix.

@sec
Copy link
Contributor

sec commented Sep 11, 2024

The fix is not yet into port that install dotnet from pkg. you can set DOTNET_ROOT to /usr/local/dotnet for your user/session for ex. to have it resolved until now.

@arrowd
Copy link
Contributor

arrowd commented Sep 11, 2024

Taking an opportunity to remind that a proper port maintainer would be great to have.
I can do review and gate changes in, but I don't have time to fully maintain the port.

@greggyb
Copy link

greggyb commented Sep 11, 2024

I have never maintained a port for any system, and the build process here is a bit beyond me.

Nevertheless, I am happy to help or take on maintainership with some guidance. We target dotnet on FreeBSD, so this is important to have and worth dedicating work hours to.

Given my ignorance mentioned above, would it be worthwhile for me to pursue this?

@nkosi23
Copy link

nkosi23 commented Sep 12, 2024

@greggyb If you want a brutal crash course on port maintenance and enjoy complex technical challenges, you can take a shot but be prepared for a lot of learning...

In all honestly I do not think that maintaining the dotnet port is beginner-friendly. Beyond getting around understanding the port system, you also need to be able to debug and fix stuff TheFrank and sec style when things break deep into the the build process because of upstream changes.

I've personally struggled to create a port for the ydb database despite the fact that it essentially can be built as is on Unix, just understanding the port system is a task on its own. If you add to this the complexity of building dotnet, it could be overwhelming.

That being said, I've heard about folks who maintain hundreds of ports like it's a walk in the park. So maybe I'm just not wired for this but you very well may be.

Just my 2 cents!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-Meta enhancement Product code improvement that does NOT require public API changes/additions needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration os-freebsd FreeBSD OS
Projects
No open projects
Status: Untriaged
Development

Successfully merging a pull request may close this issue.