Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System.ArgumentException: Unknown framework version 7.0 #987

Closed
jakubch1 opened this issue Jun 24, 2022 · 37 comments · Fixed by #989 or OctopusDeploy/Halibut#223
Closed

System.ArgumentException: Unknown framework version 7.0 #987

jakubch1 opened this issue Jun 24, 2022 · 37 comments · Fixed by #989 or OctopusDeploy/Halibut#223

Comments

@jakubch1
Copy link

jakubch1 commented Jun 24, 2022

Hello,
I'm getting System.ArgumentException: Unknown framework version 7.0 when trying to use nunit on .net framework project:

PS D:\devdiv\vscodecoverage> dotnet --version
7.0.100-preview.4.22252.9
PS D:\devdiv\vscodecoverage> dotnet test .\artifacts\bin\Microsoft.CodeCoverage.Console.Common.Tests\Release\net48\Microsoft.CodeCoverage.Console.Common.Tests.dll
Microsoft (R) Test Execution Command Line Tool Version 17.3.0-preview-20220414-05 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Exception System.TypeInitializationException,    Exception thrown executing tests in D:\devdiv\vscodecoverage\.\artifacts\bin\Microsoft.CodeCoverage.Console.Common.Tests\Release\net48\Microsoft.CodeCoverage.Console.Common.Tests.dll
The type initializer for 'NUnit.Engine.Services.RuntimeFrameworkService' threw an exception.
   at NUnit.Engine.Services.RuntimeFrameworkService.ApplyImageData(TestPackage package)
   at NUnit.Engine.Services.RuntimeFrameworkService.SelectRuntimeFramework(TestPackage package)
   at NUnit.Engine.Runners.MasterTestRunner.GetEngineRunner()
   at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnitEngine.NUnitEngineAdapter.Explore(TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\NUnitEngineAdapter.cs:line 88
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, IGrouping`2 testCases, TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 275
InnerException: System.ArgumentException: Unknown framework version 7.0
Parameter name: version
   at NUnit.Engine.RuntimeFramework.GetClrVersionForFramework(Version frameworkVersion)
   at NUnit.Engine.RuntimeFramework..ctor(RuntimeType runtime, Version version, String profile)
   at NUnit.Engine.RuntimeFramework.GetNetCoreRuntimesFromDirectoryNames(IEnumerable`1 dirNames)
   at NUnit.Engine.RuntimeFramework.FindDotNetCoreFrameworks()
   at NUnit.Engine.RuntimeFramework.FindAvailableFrameworks()
   at NUnit.Engine.RuntimeFramework.get_AvailableFrameworks()
   at NUnit.Engine.Services.RuntimeFrameworkService..cctor()
No test is available in D:\devdiv\vscodecoverage\.\artifacts\bin\Microsoft.CodeCoverage.Console.Common.Tests\Release\net48\Microsoft.CodeCoverage.Console.Common.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

Additionally, path to test adapters can be specified using /TestAdapterPath command. Example  /TestAdapterPath:<pathToCustomAdapters>.

This is probably related to nunit/nunit-console#1176 but I'm using vstest.console and adapter.

I'm using:

    <NUnitVersion>3.13.3</NUnitVersion>
    <NUnitTestAdapterVersion>4.2.1</NUnitTestAdapterVersion>
@mikkelbu
Copy link
Member

@jakubch1 The adapter uses the Nunit engine that comes from the nunit-console project, so I'm moving it there. The issue you linked to is fixed in version 4.0 of the engine, but this has not been released yet, and when it has been released, then the adapter needs to be updated to use the new engine.

Have you tried the suggestion in nunit/nunit-console#1176 (comment) ?

@mikkelbu mikkelbu transferred this issue from nunit/nunit Jun 24, 2022
@jakubch1
Copy link
Author

nunit/nunit-console#1176 (comment) is not working for us.
Any idea when you will release new version?

@OsirisTerje
Copy link
Member

@jakubch1 I'll get out a pre-release version with the 4.0 dev version of the engine during the weekend.

@OsirisTerje
Copy link
Member

@moh-hassan
Copy link

moh-hassan commented Jun 26, 2022

In multi target net50, net45 project,
I get the same error in net45 when trying to move from NUnit3TestAdapter version 4.1.0 to v4.2.0 and above.
It is working fine in net 50 with NUnit3TestAdapter version v4.2.0.

Click to show the error details!
 ========== Starting test discovery ==========
NUnit Adapter 4.2.0.0: Test discovery starting
Exception System.TypeInitializationException, Exception thrown discovering tests in C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Debug\net45\MyProject.Test.dll
The type initializer for 'NUnit.Engine.Services.RuntimeFrameworkService' threw an exception.
   at NUnit.Engine.Services.RuntimeFrameworkService.ApplyImageData(TestPackage package)
   at NUnit.Engine.Services.RuntimeFrameworkService.SelectRuntimeFramework(TestPackage package)
   at NUnit.Engine.Runners.MasterTestRunner.GetEngineRunner()
   at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnitEngine.NUnitEngineAdapter.Explore(TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\NUnitEngineAdapter.cs:line 88
   at NUnit.VisualStudio.TestAdapter.NUnit3TestDiscoverer.DiscoverTests(IEnumerable`1 sources, IDiscoveryContext discoveryContext, IMessageLogger messageLogger, ITestCaseDiscoverySink discoverySink) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestDiscoverer.cs:line 82
InnerException: System.ArgumentException: Unknown framework version 7.0
Parameter name: version
   at NUnit.Engine.RuntimeFramework.GetClrVersionForFramework(Version frameworkVersion)
   at NUnit.Engine.RuntimeFramework..ctor(RuntimeType runtime, Version version, String profile)
   at NUnit.Engine.RuntimeFramework.GetNetCoreRuntimesFromDirectoryNames(IEnumerable`1 dirNames)
   at NUnit.Engine.RuntimeFramework.FindDotNetCoreFrameworks()
   at NUnit.Engine.RuntimeFramework.get_AvailableFrameworks()
   at NUnit.Engine.Services.RuntimeFrameworkService..cctor()
NUnit Adapter 4.2.0.0: Test discovery complete
No test is available in C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Debug\net45\MyProject.Test.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
NUnit Adapter 4.2.0.0: Test discovery starting
NUnit Adapter 4.2.0.0: Test discovery complete
========== Test discovery finished: 1 Tests found in 2.9 sec ==========
Building Test Projects
========== Starting test run ==========
========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in < 1 ms ==========
Building Test Projects
========== Starting test run ==========
NUnit Adapter 4.2.0.0: Test execution started
Running selected tests in C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Debug\net5.0\MyProject.Test.dll
   NUnit3TestExecutor discovered 1 of 1 NUnit test cases using Current Discovery mode, Non-Explicit run
NUnit Adapter 4.2.0.0: Test execution complete
========== Test run finished: 1 Tests (1 Passed, 0 Failed, 0 Skipped) run in 196 ms ==========
Building Test Projects
========== Starting test run ==========
========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in < 1 ms ==========

Demo Project to show the issue
MyProject.Test.zip

@OsirisTerje
Copy link
Member

@moh-hassan 1. This is not the net 7 issue, is it ? 2. Testing your repro, it all works on my machine, both in VS2022 and with dotnet test.

@jakubch1
Copy link
Author

@OsirisTerje thanks for new version. Unfortunately, it is still not working (for .NET Framework). Old error is gone but I see in logs:

Line 778: TpTrace Verbose: 0 : 22028, 12, 2022/06/27, 09:14:04.751, 2391146649367, testhost.net48.x86.exe, TestRequestHandler.SendData:  sending data from testhost: {"Version":6,"MessageType":"TestSession.Message","Payload":{"MessageLevel":0,"Message":"   NUnit failed to load D:\\devdiv\\vscodecoverage\\artifacts\\bin\\Microsoft.CodeCoverage.IO.Tests\\Release\\net48\\.\\Microsoft.CodeCoverage.IO.Tests.dll"}}
*** Assembly Binder Log Entry  (6/27/2022 @ 10:16:07 AM) ***

The operation failed.
Bind result: hr = 0x80131045. No description available.

Assembly manager loaded from:  C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll
Running under executable  C:\Program Files\dotnet\sdk\7.0.100-preview.4.22252.9\TestHost\testhost.net48.x86.exe
--- A detailed error log follows. 

=== Pre-bind state information ===
LOG: Where-ref bind. Location = D:\devdiv\vscodecoverage\artifacts\bin\Microsoft.CodeCoverage.IO.Tests\Debug\net48\Microsoft.CodeCoverage.IO.Tests.dll
LOG: Appbase = file:///D:/devdiv/vscodecoverage/artifacts/bin/Microsoft.CodeCoverage.IO.Tests/Debug/net48
LOG: Initial PrivatePath = NULL
LOG: Dynamic Base = NULL
LOG: Cache Base = C:\Users\UseThisAccount\AppData\Local\Temp\93736a70-0c12-4762-bc21-188b2052bc6d
LOG: AppName = 93736a70-0c12-4762-bc21-188b2052bc6d
Calling assembly : (Unknown).
===
LOG: This is an inspection only bind.
LOG: Using application configuration file: C:\Program Files\dotnet\sdk\7.0.100-preview.4.22252.9\TestHost\testhost.net48.x86.exe.Config
LOG: Using host configuration file: 
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.30319\config\machine.config.
LOG: Attempting download of new URL file:///D:/devdiv/vscodecoverage/artifacts/bin/Microsoft.CodeCoverage.IO.Tests/Debug/net48/Microsoft.CodeCoverage.IO.Tests.dll.
LOG: Assembly download was successful. Attempting setup of file: D:\devdiv\vscodecoverage\artifacts\bin\Microsoft.CodeCoverage.IO.Tests\Debug\net48\Microsoft.CodeCoverage.IO.Tests.dll
LOG: Entering download cache setup phase.
LOG: Assembly Name is: Microsoft.CodeCoverage.IO.Tests, Version=42.42.42.42, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
ERR: Setup failed with hr = 0x80131045.
ERR: Failed to complete setup of assembly (hr = 0x80131045). Probing terminated.

@moh-hassan
Copy link

@OsirisTerje

  1. This is not the net 7 issue, is it ?

yes, It's FullFramework issue, .NET45 up to .NET48
The issue is only with NUnit3TestAdapter version v4.2.0 and above. It is fine in lower version NUnit3TestAdapter version v4.1.0.

The error is
InnerException: System.ArgumentException: Unknown framework version 7.0
In netcore net5.0, net6.0 it's Fine with all NUnit3TestAdapter versions.

I used global.json to control dotnet and it has no effect.

I have the next sdk versions installed including net7 preview5

dotnet --list-sdks

2.0.0 [C:\Program Files\dotnet\sdk]
2.1.818 [C:\Program Files\dotnet\sdk]
3.1.420 [C:\Program Files\dotnet\sdk]
5.0.403 [C:\Program Files\dotnet\sdk]
5.0.408 [C:\Program Files\dotnet\sdk]
6.0.100 [C:\Program Files\dotnet\sdk]
6.0.106 [C:\Program Files\dotnet\sdk]
6.0.202 [C:\Program Files\dotnet\sdk]
6.0.203 [C:\Program Files\dotnet\sdk]
6.0.301 [C:\Program Files\dotnet\sdk]
7.0.100-preview.5.22307.18 [C:\Program Files\dotnet\sdk]

The error is the same, either using vs 2022 v 17.2.5 or using dotnet test.

@moh-hassan
Copy link

@OsirisTerje
I uninstalled net7 preview5, and now it's working fine in Net45..net48 with NUnit3TestAdapter version v4.2.1.
It seems that net7 preview5 is responsible for this error although I use global.json with setting net5.

@OsirisTerje
Copy link
Member

@jakubch1 Yes, I do see the same error here.
@CharliePoole Can you shed some more light on this error? It must be the engine struggling here. It also happens just when using net4.5 as target.

I've added @moh-hassan 's repro above to the repro repository, and updated with the new package, so you can test it with the code here: https://github.com/nunit/nunit3-vs-adapter.issues/tree/master/Issue987/MyProject.Test

The dump file from the discovery looks like this

<?xml version="1.0" encoding="utf-8"?>
<NUnitXml>
<test-run id='0' runstate='Runnable' testcasecount='1' result='Failed' label='Error' total='1' passed='0' failed='1' warnings='0' inconclusive='0' skipped='0' asserts='0'>
   <test-suite type='Assembly' id='0' name='' fullname='' runstate='NotRunnable' testcasecount='1' result='Failed' label='Error' start-time='2022-06-27 14:30:13Z' end-time='2022-06-27 14:30:13Z' duration='0.001' total='1' passed='0' failed='1' inconclusive='0' skipped='0' asserts='0'>
      <failure>
         <message><![CDATA[System.ComponentModel.Win32Exception : The system cannot find the file specified]]></message>
         <stack-trace><![CDATA[--Win32Exception
The system cannot find the file specified
   at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo)
   at NUnit.Engine.Services.TestAgency.GetAgent(TestPackage package)
   at NUnit.Engine.Runners.ProcessRunner.CreateAgentAndRunnerIfNeeded()
   at NUnit.Engine.Runners.ProcessRunner.Explore(TestFilter filter)
]]></stack-trace>
      </failure>
   </test-suite>
</test-run>
</NUnitXml>
```

@CharliePoole Name and Fullname is not there, so is the error related to that?

@OsirisTerje
Copy link
Member

@jakubch1 @moh-hassan I found the missing link in the engine, and have uploaded a new adapter package https://www.nuget.org/packages/NUnit3TestAdapter/4.3.0-alpha-net7.4

The Issue project above now works:

image

Please verify this with your solutions.

@CharliePoole I'll upload a PR to the console/engine project rsn.

@OsirisTerje
Copy link
Member

This fix was based on the master branch, which is back on 3.13, .... it works, but I have to do the same for the latest codebase. The trunk is now called main for the engine project...... So much for that change .......

@moh-hassan
Copy link

moh-hassan commented Jun 27, 2022

@OsirisTerje
Thanks for the new version and fix.
I test NUnit3TestAdapter" Version="4.3.0-alpha-net7.4" , and it and it's working fine in net45..net48, net5.0, net6.0, net7.0, even with using net7 7.0.100-preview.5.22307.18. 👍

nunit2

Also, It's working fine using CLI dotnet test

Click to show dotnet test!
 

  Determining projects to restore...
  All projects are up-to-date for restore.
C:\Program Files\dotnet\sdk\7.0.100-preview.5.22307.18\Sdks\Microsoft.NET.Sdk\targets
\Microsoft.NET.RuntimeIdentifierInference.targets(219,5): message NETSDK1057: You are
 using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [C:\Users
\mohha\source\repos\MyProject.Test\MyProject.Test\MyProject.Test.csproj]
C:\Program Files\dotnet\sdk\7.0.100-preview.5.22307.18\Sdks\Microsoft.NET.Sdk\targets
\Microsoft.NET.RuntimeIdentifierInference.targets(219,5): message NETSDK1057: You are
 using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [C:\Users
\mohha\source\repos\MyProject.Test\MyProject.Test\MyProject.Test.csproj]
C:\Program Files\dotnet\sdk\7.0.100-preview.5.22307.18\Sdks\Microsoft.NET.Sdk\targets
\Microsoft.NET.RuntimeIdentifierInference.targets(219,5): message NETSDK1057: You are
 using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [C:\Users
\mohha\source\repos\MyProject.Test\MyProject.Test\MyProject.Test.csproj]
C:\Program Files\dotnet\sdk\7.0.100-preview.5.22307.18\Sdks\Microsoft.NET.Sdk\targets
\Microsoft.NET.RuntimeIdentifierInference.targets(219,5): message NETSDK1057: You are
 using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [C:\Users
\mohha\source\repos\MyProject.Test\MyProject.Test\MyProject.Test.csproj]
C:\Program Files\dotnet\sdk\7.0.100-preview.5.22307.18\Sdks\Microsoft.NET.Sdk\targets
\Microsoft.NET.RuntimeIdentifierInference.targets(219,5): message NETSDK1057: You are
 using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [C:\Users
\mohha\source\repos\MyProject.Test\MyProject.Test\MyProject.Test.csproj]
C:\Program Files\dotnet\sdk\7.0.100-preview.5.22307.18\Sdks\Microsoft.NET.Sdk\targets
\Microsoft.NET.RuntimeIdentifierInference.targets(219,5): message NETSDK1057: You are
 using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [C:\Users
\mohha\source\repos\MyProject.Test\MyProject.Test\MyProject.Test.csproj]
C:\Program Files\dotnet\sdk\7.0.100-preview.5.22307.18\Sdks\Microsoft.NET.Sdk\targets
\Microsoft.NET.RuntimeIdentifierInference.targets(219,5): message NETSDK1057: You are
 using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [C:\Users
\mohha\source\repos\MyProject.Test\MyProject.Test\MyProject.Test.csproj]
  MyProject.Test -> C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Deb
ug\net45\MyProject.Test.dll
  MyProject.Test -> C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Deb
ug\net48\MyProject.Test.dll
  MyProject.Test -> C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Deb
ug\net472\MyProject.Test.dll
  MyProject.Test -> C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Deb
ug\net461\MyProject.Test.dll
  MyProject.Test -> C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Deb
ug\net5.0\MyProject.Test.dll
  MyProject.Test -> C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Deb
ug\net7.0\MyProject.Test.dll
  MyProject.Test -> C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Deb
ug\net6.0\MyProject.Test.dll
Test run for C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Debug\net7
.0\MyProject.Test.dll (.NETCoreApp,Version=v7.0)
Microsoft (R) Test Execution Command Line Tool Version 17.3.0-preview-20220523-03 (x6
4)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 33 m
s - MyProject.Test.dll (net7.0)
Test run for C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Debug\net6
.0\MyProject.Test.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.3.0-preview-20220523-03 (x6
4)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 27 m
s - MyProject.Test.dll (net6.0)
Test run for C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Debug\net5
.0\MyProject.Test.dll (.NETCoreApp,Version=v5.0)
Microsoft (R) Test Execution Command Line Tool Version 17.3.0-preview-20220523-03 (x6
4)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 29 m
s - MyProject.Test.dll (net5.0)
Test run for C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Debug\net4
5\MyProject.Test.dll (.NETFramework,Version=v4.5)
Microsoft (R) Test Execution Command Line Tool Version 17.3.0-preview-20220523-03 (x6
4)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 108 
ms - MyProject.Test.dll (net45)
Test run for C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Debug\net4
72\MyProject.Test.dll (.NETFramework,Version=v4.7.2)
Microsoft (R) Test Execution Command Line Tool Version 17.3.0-preview-20220523-03 (x6
4)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 108 
ms - MyProject.Test.dll (net472)
Test run for C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Debug\net4
8\MyProject.Test.dll (.NETFramework,Version=v4.8)
Microsoft (R) Test Execution Command Line Tool Version 17.3.0-preview-20220523-03 (x6
4)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 108 
ms - MyProject.Test.dll (net48)
Test run for C:\Users\mohha\source\repos\MyProject.Test\MyProject.Test\bin\Debug\net4
61\MyProject.Test.dll (.NETFramework,Version=v4.6.1)
Microsoft (R) Test Execution Command Line Tool Version 17.3.0-preview-20220523-03 (x6
4)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 105 
ms - MyProject.Test.dll (net461)


@jakubch1
Copy link
Author

4.3.0-alpha-net7.4 is working fine. Thanks a lot for help.

@CharliePoole
Copy link
Contributor

Sorry to be late for the party. Since I don't work on the adapter any longer, I don't usually see issues that arise. Also, I'm on vacation. :-)

The engine does not yet support tests, which target .NET 7, but that does not seem to be the issue here. It is supposed to run .NET 5.0 tests successfully, even if the only version installed is 7.0.

However, in order to run at all, the engine needs to find out what runtimes are installed on the machine. It does that before it even tries to load your tests, by looking at the names of install directories. We have had problems in the past with preview releases creating directories with odd names. This was fixed in 3.15, by ignoring directories, whose names did not have the form of a version number. If you only have a preview release with a non-standard directory name, then the engine will think you have no runtimes capable of handling .NET 5 or .NET 6 as well.

OTOH if you have (for example) .NET 6 installed instead, it should work.

Currently, I test possible engine problems using a sequence like this...

  1. If I think it may be a framework problem, I run the tests using NUnitLite, so the engine isn't involved.
  2. If they run all right under NUnitLite, I test with the console. The console obviously adds things to the engine, but it is closer to it and many things are simply passed through to the user directly. I can debug in the engine to see what's happening, up to the point where a separate process is launched.
  3. If the problem is in the agent process, I have to use a pretty tedious debugging approach, by attaching to the agent. This works but takes a lot of time.
  4. Note that I do not mention TestCentric GUI, which I see mentioned in some other threads. That is because the GUI uses the TestCentric engine, which is a different engine from NUnit's. That's something to be resolved eventually, but the resolution is complicated because the GUI went a different way during the years when I didn't work on NUnit at all. So, as a general rule, please don't try to understand what NUnit is doing by using the GUI.

@CharliePoole
Copy link
Contributor

@OsirisTerje Can you briefly summarize the status of this from your point of view? Do you have a private build based on 3.15? Do you want to do a PR? Or if it's a small change, I can just copy it.

I had not planned to do any releases after 3.15, because I thought 4.0 would be quicker than it has turned out to be. Do you think we should somehow be coordinating releases? We'd have to decide exactly how... For example. do you only depend on final version releases of the engine? Do we use a set of acceptance tests for the adapter, which are known to pass with the older version of the engine? Other stuff?

@CharliePoole
Copy link
Contributor

One final note for those testing issues like this one: Make sure you are not referencing two versions of the engine! If you reference both the console runner and the adapter, you must know which versions of the engine each of them uses because one of the two will "win" and be used by both runners. I wrote a post about this quite a long time ago, but it may not be up to date. Maybe this info could be included in the adapter documentation and kept up to date.

See http://charliepoole.org/technical/nunit-engine-version-conflicts-in-visual-studio.html

@OsirisTerje
Copy link
Member

OsirisTerje commented Jun 28, 2022

  1. The problem arise when people install .net 7 (which is in preview now)

  2. The code for engine 3.13.0 works – when the switch statement in RunTimeFramework.cs (approx. line 151) is extended with version 7:
    case 7:
    return new Version(7, 0, 0);
    Any later tag I have tried crashes.

  3. If I try the latest dev build (xxxx102) from myget, then it crashes with a “Win32 error” saying it can’t find the the file specified, see comment System.ArgumentException: Unknown framework version 7.0 #987 (comment) .

  4. I found that the crash happens from the GetAgent method, and further down in the constructor of a class named ExtensionAssembly which is being used by the GetAgent method.

image

  1. Use the repro project at https://github.com/nunit/nunit3-vs-adapter.issues/tree/master/Issue987/MyProject.Test and just switch in this adapter: NuGet Gallery | NUnit3TestAdapter 4.3.0-alpha-net7, this uses the latest dev version.
  2. Got to add: It crashes when one uses .net 4.8 or lower. Works when using .net 5 or higher, haven’t tested the other lower netcores.

@CharliePoole
Copy link
Contributor

A few comments using your numbering for clarity...

1 and 2. The same fix is present in PR 1175, which was merged on March 23. This is of course subsequent to the 3.15 release, so it has never actually been released.

  1. The full version number you are referencing is 4.0.0-dev00102. That is, it's a 4.0.0 dev release, containing tons of breaking changes in the API. You should be prepared for a lot of work if you intend to switch to the 4.0.0 engine build. I wouldn't advise it until the API is stable. Even then, I suggest using alpha releases rather than dev releases. There are no 3.x dev builds because I had intended 3.15 to be the last 3.x release. Full .NET 7.0 support (i.e. tests referencing 7.0) is planned only for 4.0.

  2. The method is trying to decide what agent to use for the particular assembly. IIRC the use of ExtensionAssembly here is new in the 4.x builds and related to a lot of other changes. See 3.

  3. As stated before, I think using the dev 4.0 builds, with breaking changes, for your alpha builds is risky. IMO, you should only use actual release builds and we should backport the 4.0 fix to create a new 3.15.1 build, provided that small change is all you really need.

  4. As mentioned elsewhere, errors can occur simply searching for available frameworks. In addition, they can occur in loading tests or extensions. Note that an agent may be an extension.

I'll create an issue to backport the fix in PR 1175 to 3.15.1.

@CharliePoole
Copy link
Contributor

FYI - I created a release-3.15.1 branch of the console and engine, based on the 3.15 release.

@CharliePoole
Copy link
Contributor

The 3.15.1-dev00003 release on MyGet should handle this problem.

@CharliePoole
Copy link
Contributor

@OsirisTerje Are you adding info to the docs about the engine version in use for new releases?

@OsirisTerje
Copy link
Member

Yes, I will do that. Might perhaps also add a table in the docs for this, could be helpful going forward.

@CharliePoole
Copy link
Contributor

Some info is collected in my old blog post on charliepoole.org.

@OsirisTerje
Copy link
Member

Right! That table should be in the adapter docs, which would make it easier to maintain it, or I can point to that blog if you can update it ?

@CharliePoole
Copy link
Contributor

Probably better to take over the info. I'll do a final update to that post pointing to your copy.

@OsirisTerje
Copy link
Member

@jakubch1 Version 4.3.0 with .net 7 support is released now

@CharliePoole
Copy link
Contributor

@OsirisTerje So does the fact that you are running in-process allow .NET 7.0 tests to actually execute under 4.3.0?

If so, that's a big advantage to letting VS set up the process being used to run tests. In any case, the 3.16 engine will have an agent for 7.0 but the proliferation of agents is getting a bit out of hand. :-(

@OsirisTerje
Copy link
Member

@CharliePoole Yes, it works fine with .net 7 both inside VS 2022 version 17.4 and from dotnet cmd line.
image

@indy-singh
Copy link

Is this fixed? We are starting to see this error as machines get upgraded to dotnet7.0.

Our test project is on .net framework 4.7.2.

@OsirisTerje
Copy link
Member

Yes, see #1026 and #1027. There is an alpha version enclosed with those. Please check that one out and confirm if it works for you. Will try to make a hotfix release within a few days now

@CharliePoole
Copy link
Contributor

@OsirisTerje I think I'll be doing a beta release of the 3.16 engine within a few days, in case you want to wait for that.

@OsirisTerje
Copy link
Member

@CharliePoole Yes, think that will be wise.

@moh-hassan
Copy link

moh-hassan commented Nov 14, 2022

I think it is better to follow the advice in documentation::)

The NUnit3TestAdapter is the best place to start.

Nunit is working fine using NUnit3TestAdapter v4.3.0 in net7 and old frameworks.

@OsirisTerje
Copy link
Member

@moh-hassan Thanks for the confirmation!

@bxm185
Copy link

bxm185 commented Nov 30, 2022

Upgrade NUnit3TestAdapter to 4.3.1 and the tests should work

scott-xu added a commit to scott-xu/ironpython3 that referenced this issue Dec 6, 2022
Fix "System.ArgumentException: Unknown framework version 7.0" in step Test (net462)
Reference: nunit/nunit3-vs-adapter#987
slozier pushed a commit to IronLanguages/ironpython3 that referenced this issue Dec 6, 2022
* Update IronPythonTest.csproj

Fix "System.ArgumentException: Unknown framework version 7.0" in step Test (net462)
Reference: nunit/nunit3-vs-adapter#987

* Update IronPythonTest.csproj

NUnit3TestAdapter 4.3.1 dropped netcoreapp2.1
nathanwoctopusdeploy added a commit to OctopusDeploy/Halibut that referenced this issue Apr 13, 2023
ForNeVeR added a commit to JetBrains/rd that referenced this issue Dec 2, 2023
ForNeVeR added a commit to JetBrains/rd that referenced this issue Dec 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants