Unable to run tests on FreeBSD

[sec]

Hi,
I've compiled CoreCLR and CoreFX (both managed and native parts). Also did build-tests.cmd and copied them over to FreeBSD box. Now when trying to run the tests I get errors like:

Cannot find /home/sec/core/corefx/bin/tests/Canonicalization.Tests/netcoreapp-FreeBSD-Debug-x64/RunTests.sh

I'm missing something? I tried reading docs, but they seem little not up-to-date?

[karelz]

Which docs did you read? What is not up-to-date?
Running the test did not start yet AFAIK and is not documented on wiki. cc @wfurt
I don't think @wfurt

[wfurt]

check if the script has executable flag @sec

I'm in process getting that ready. I had troubles with the test version as well as there is something broken badly broken as most tests simply fail on me with System.NullReferenceException : Object reference not set to an instance of an object.

Also System.Console did not work right so I was getting very little output from test run.
My goal was to get test runs to somewhat stable state and post instructions and stats on FreeBSD wiki page.

[sec]

@karelz Hi. I've read https://github.com/dotnet/corefx/wiki/Build-and-run-tests and https://github.com/dotnet/coreclr/blob/master/Documentation/building/unix-test-instructions.md
I didn't read the wiki page you linked, I know it's a work in the progress (coming from https://github.com/dotnet/corefx/issues/1626).
What I did was, build tests on Windows box, copyied them to FreeBSD and I've tried to run them using binaries compiled on FreeBSD - maybe I did something wrong here, please don't blame me :), I'm getting to know this build system.

@wfurt The problem is, that this script don't exist and I can't find it either on Windows box or FreeBSD one.

@wfurt Also regarding that wiki page - maybe I can help you update it, some of the things can be simplified (for ex. freebsd corelib can be build on windows with 1 command, coreclr/corefx build native parts without any hacks from master as for now, corefx managed can be build on windows (there's 3 or 4 dll not supported still like FileWatcher, dotnet cli can also be build on FreeBSD, etc.). I think we're missing the part on how to get all those things together to form "dotnet runtime with dll's etc".

[wfurt]

no blame @sec :) Updating documentation always good.
I used combination of Linux & FreeBSD - as it seemed more natural and it was easier for me to copy files around and do troubleshooting. I have dotnet/coreclr#14121 but the idea of single command to build corelib is being rejected. BTW what script are you referring to? If it is RunTests.sh, did you try any other for different assemblies?

I made some progress on getting tests running. I'm planning to run them all, get stats and post some instructions today. I also start filing issues for missing or broken parts.

As far as dotnet cli I have private bastard build with full sdk functionality.
However that needs more fixes beyond basic repos. It can run simple assemblies but when running msbuild I get some assert failures in coreclr. I'm trying to chase that down but I'd be happy to pass any chunks of work to anybody from community. I have PR pending for core-setup and once that is in, I can pass instructions. There are generally two approaches: One is to grab some dotnetcli and replace needed parts with FreeBSD bits. I used that to build my private bastard. Full build from sources is more complicated. I'm ignoring that for now and I'm hoping that work for source-build repo will pay back. Even if their focus is on Linux, once done, it will pave way for getting full build reasonable.

Let me know what do you think @sec There is still plenty of work but every improvement counts.
Let me know what are interests you most.

[sec]

@wfurt I saw in #14537 that you were able to execute CoreFX tests, so I think I'm gonna wait for your instructions how you did it and then see on which parts I can help with

[wfurt]

ok @sec. I started yesterday on docs but I did not finish.
My build trees are in somewhat messy state polluted with bunch of changes.
I created new pristine system as well as I would like to include coreclr.
That slowed me down somewhat. I'l update this issue when docs are ready.

[wfurt]

I updated instructions how to run corefx tests @sec and @RussellHaley
It would be great if you can give it try and let me know.
First round is little bit tedious but once done, iterations on fixing code is not that bad.

Note, that XUnits has problem with output because of missing terminfo.
I simply copy terminfo database to /etc for now.

[sec]

@wfurt Hi, I started to build using new code from all of dotnet repo's. Quick status:

1. coreclr didn't build on freebsd (due to dotnet/coreclr@08d39dd) - when I reverted this commit, it builds. cc @janvorli for the error below (simple change from NULL to 0 fixed those errors, but it was in many places, so I just reverted whole commit, also I'm not a c++ expert). ci build for freebsd would be nice to have :)
   [ 3%] Building CXX object src/vm/dac/CMakeFiles/cee_dac.dir/__/appdomain.cpp.o /home/sec/core/coreclr/src/pal/src/init/pal.cpp:292:24: error: no matching function for call to 'InterlockedCompareExchangePointerT' if(NULL != InterlockedCompareExchangePointer(&init_critsec, &temp_critsec, NULL)) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/sec/core/coreclr/src/pal/src/include/pal/palinternal.h:708:43: note: expanded from macro 'InterlockedCompareExchangePointer' #define InterlockedCompareExchangePointer InterlockedCompareExchangePointerT ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/sec/core/coreclr/src/pal/src/include/pal/palinternal.h:696:11: note: candidate function [with T = _CRITICAL_SECTION] not viable: no known conversion from 'nullptr_t' to 'int' for 3rd argument inline T* InterlockedCompareExchangePointerT( ^ /home/sec/core/coreclr/src/pal/src/include/pal/palinternal.h:665:4: note: candidate template ignored: could not match 'T *' against 'nullptr_t' T* InterlockedCompareExchangePointerT( ^ /home/sec/core/coreclr/src/pal/src/include/pal/palinternal.h:686:11: note: candidate template ignored: could not match 'T *' against 'nullptr_t' inline T* InterlockedCompareExchangePointerT( ^ 1 error generated. --- src/pal/src/CMakeFiles/coreclrpal.dir/init/pal.cpp.o --- *** [src/pal/src/CMakeFiles/coreclrpal.dir/init/pal.cpp.o] Error code 1
2. coreclr freebsdmscorlib was built on windows
3. corefx build without errors (native on freebsd, managed on windows)
4. core-setup (dotnet cli) also builds

Next I will move to building tests and running them.

Edit 1 - coreclr tests result:
Finished running PAL tests.

The following test(s) failed:
threading/NamedMutex/test1/paltest_namedmutex_test1. Exit code: 1

PAL Test Results:
Passed: 762
Failed: 1

[wfurt]

the build problem should be fixed in master @sec I have changes in my private drop to make more corefx tests runable.

[wfurt]

I was able to run all the tests sucessfully with current master:

Discovering: Canonicalization.Tests
Discovered:  Canonicalization.Tests
Starting:    Canonicalization.Tests
   System.Runtime.Serialization.Xml.Canonicalization.Tests.XmlCanonicalizationTest.CryptoCanonicalizationTest [STARTING]
   System.Runtime.Serialization.Xml.Canonicalization.Tests.XmlCanonicalizationTest.CryptoCanonicalizationTest [FINISHED] Time: 0.5663712s
   System.Runtime.Serialization.Xml.Canonicalization.Tests.XmlCanonicalizationTest.C14NWriterNegativeTests [STARTING]
   System.Runtime.Serialization.Xml.Canonicalization.Tests.XmlCanonicalizationTest.C14NWriterNegativeTests [FINISHED] Time: 0.0043241s
   System.Runtime.Serialization.Xml.Canonicalization.Tests.XmlCanonicalizationTest.ReaderWriter_C14N_DifferentReadersWriters [STARTING]
   System.Runtime.Serialization.Xml.Canonicalization.Tests.XmlCanonicalizationTest.ReaderWriter_C14N_DifferentReadersWriters [FINISHED] Time: 0.1366794s
   System.Runtime.Serialization.Xml.Canonicalization.Tests.XmlCanonicalizationTest.TestC14NInclusivePrefixes [STARTING]
   System.Runtime.Serialization.Xml.Canonicalization.Tests.XmlCanonicalizationTest.TestC14NInclusivePrefixes [FINISHED] Time: 0.0040945s
Finished:    Canonicalization.Tests

=== TEST EXECUTION SUMMARY ===
   Canonicalization.Tests  Total: 4, Errors: 0, Failed: 0, Skipped: 0, Time: 0.943s
~/git/wfurt-corefx/bin/tests/Canonicalization.Tests/netcoreapp-FreeBSD-Debug-x64

please reopen if you still have problems @sec