-
Notifications
You must be signed in to change notification settings - Fork 154
Adapter Release Notes
This release is a hotfix release intended to fix three major issues:
- 686 NUnit3TestAdapter3.16. dotnet filter argument is not applied
- 690 Can run only one test case from combinatorial test
- 692 NUnit Test Adaptor 3.16.0 never displays results of test under Visual Studio 2015
Together with these, the following issues are changed:
- 676 Test cases are skipped with TestCaseSource under Visual Studio 2019. This works under 3.16.0, but need the changes below in a runsettings file under 3.16.1
Note that this hotfix changes some of the new defaults introduced in 3.16.0.
These can be set back using the two new runsettings UseParentFQNForParametrizedTests and UseNUnitIdforTestCaseId
This release has three major changes.
1 The support for .Net Core 1.* has been removed. This is done to open up more functionality for the later .net core versions. Also, since .net core 1.* is no longer supported by Microsoft, we decided to remove the support too. If you do have .net core 1.* solutions and can't upgrade, you should stay with the the adapter 3.15.1 or lower. The change also means we now support dump files in .net core :-). It also means the total size of the package has been reduced by around 30%.
2 The filter syntax issues we've had with special names and characters have been mostly solved, thanks to excellent work by John M.Wright. The filter syntax is now closer to a correct FQN (Full Qualified Name), but this might cause some issues with your own filter in some rare cases. The fix done resolves a lot of issues, all of them listed below. For a detailed explanation of what has been done, see the Pull Request #668.
3 You can now use the NUnit filter syntax, either from command line or through settings in the runsettings file. This was due to an idea by Michael Letterle and a subsequent implementation Pull Request #669. Michael also wrote a great blogpost to explain how this works, and how he arrived at the solution. For more information see the NUnit Test Selection Language
All the following are solved by Pull Request #668
- 622 An exception occurred while invoking executor 'executor://nunit3testexecutor/': Incorrect format for TestCaseFilter Error: Missing ')'
- 549 Invalid characters in test name breaks test runner
- 607 VS Test Explorer does not show results for test cases that have custom names
- 613 TestCaseSource tests get wrongly displayed
- 672 VS Test Explorer fails to run tests with test cases mocked with NSubstitute
- 644 Test Explorer shows excess class name for tests with multiple TestCaseAttribute
- 615 Dynamic TestCaseSource from reflection are not listed as sub-tests of the test method
The following are resolved (fully or partially) by Pull Request #669
- 655 How to run parameterized NUnit tests on .NET Core?
- 425 Partly resolved using NUnit Test Selection Language. Run only specific tests when using dotnet test?
- 582 TestCase with enclosed double quotes error
- 679 Build.cake does not work in a pure visual studio preview installation. Fixed by PR #680 by Ove Bastiansen
This is a hotfix release due to some issues with special cases in NUnit, causing the tests to not run. The major difference is that this release makes PreFiltering which was introduced in 3.15, optional - and default off. It can be enabled by setting PreFilter to true in a runsettings file.
-
651 Add featureflag to enable/disable pre-filtering. This one contains the links to the issues below:
-
648 NUnit3TestAdapter 3.15.0 fails to run test: "NUnit failed to load" (when using NUnit framework less than version 3.11)
-
649 NUnit3TestAdapter 3.15 OneTimeSetUp not working anymore (When a SetupFixture is being used)
-
650 NUnit3TestAdapter 3.15 not running tests with custom TestCaseSource (when using SetName instead of SetArgDisplayNames)
This release is a major performance improvement release. When used from Visual Studio, and used with a selection of tests, it will significantly speed up the discovery of those. Note that discovery is also a part of the execution, so you will also see the performance improvement for execution. It will be mostly noticable when you have a large set of tests.
- 529 Use framework prefilter for massive perf improvement when running fraction of tests. Thanks to MatthewBeardmore for the Pull request and patience
- 645 NUnit3TestAdapter 3.14.0 includes NUnit3.TestAdapter.dll versioned 3.13. Version number of dll corrected to match package version 3.15
- 580 Fix licenseUrl in nuspec, will be deprecated
- 609 Request to change working directory from Windows/System32 to another directory and 303 Directory.GetCurrentDirectory is C:\WINDOWS\system32 for NUnit3TestAdapter. If your currentdirectory points to either Windows directory or the Program Files folders, it will be redirected to your temp folder.
- 621 TestContext.Progress output. It will only go to the console, not the test output.
- 222 NUnit3TestAdapter errors when running xUnit tests. Thanks to nvborisenko for the Pull Request
- 629 NUnit3TestAdapter errors when running xUnit tests
- 630 Assert.Multiple produces new tests in the report if inner assertions fail. Thanks again to nvborisenko for the Pull Request
This version also includes the latest version 3.10 of the NUnit.Engine.dll
This release focuses on producing NUnit test result XML which can be useful when you need reports using tools that support the NUnit format. This can be enabled using a new setting in the runsettings file.
The VSIX is also made compatible with the upcoming VS 2019. Please note support for the VSIX is being deprecated in Visual Studio, and we strongly recommend you to change your test projects to use the NuGet adapter version.
The NUnit internal properties have been "over-populating" in the Test Explorer. These are now filtered out, although you may still see these when you have Source Based Discovery (SBD) turned on (which is the default in VS). Once you have run, they will be gone. We expect this part of the issue (SBD) to be fixed in VS.
If you still want to see the properties, you can enable that again setting a runsettings property ShowInternalProperties to true.
- 323 Add support for producing XML test results in NUnit format
- 474 Skip surfacing certain properties as UI groupings (Internal NUnit properties are no longer visible in Test Explorer)
- 590 Support for Visual Studio 2019 (VSIX)
- 302 BadImageFormatException building solution with unmanaged projects) Extra fix for this, see PR 592 Issue302 native execution in execution. Thanks to Oski Kervinen for fixing this.
- 596 TestOutputXml should handle relative paths relative to the Workfolder
- 600 TestOutputXml setting ignored on netcore2.1, works on net461
- 599 Url in the repo header pointing to nunit.org changed to https, also some other similar changes other places. Thanks to Julian Verdurmen for fixing these.
- 215 Generate NUnit xml file of test results, specified in .runsettings file for reporting
- 573 NUnit3TestDiscoverer Could not load file or assembly 'nunit.engine'
This is a yet another hotfix release to fix the missing categories issue.
- 568 NUnit3TestAdapter 3.11.1 TestCategory from VSTest no longer working (No tests selected)
This issue was fixed, with a workaround, in 3.11 but had unforeseen consequences, and has been reopened as of this release
- 506 Test categories aren't propagated to vstest trx logs
This probably need to be fixed in VSTest itself.
This is a hotfix release to fix the duplicated traits issue.
- 559 Duplicating tags in Test Explorer when using NUnit3TestAdapter 3.11.0
- 561 Test task aborted when using TestCategory filters
-
PR 500 NUnit3VsAdapter to support managed sources only based on RFC (Thanks to mayankbansal018 for this PR)
-
543 Adapter should pass
ErrorandProgressmessages to vstest engine as well as stdOut messages (Thanks to NikolayPianikov for PR 544). Also fixes this TeamCity issue -
Mono.Cecil is now embedded with the adapter, so user dependencies are no longer overwritten.
-
Indentation of the test log format makes it easier to see what information belongs to which assembly.
-
Quiet mode added, if you don't want all the information, see Tips and Tricks.
-
426 Exception thrown while loading tests if In-Proc VSTest DataCollector is used (Thanks to drognanar for PR 510 )
-
490 Fix Causes Build Error (Somewhat Indirectly) by published pdb files from Issue 461
-
494 TestContext.AddTestAttachment does not work on Linux environment with specified dotnet logger (Thanks to Kira-Lappo for PR 527)
-
495 Category as filter not working in single agent flow in vstest task
-
506 Test categories aren't propagated to vstest trx logs
-
516 ArgumentException when whitespace sent to logger
-
538 NuGet Package : Add repository metadata (Thanks to MaximRouiller for PR 539)
-
540 Missing null check before runner dispose
- 518 NUnit VSIX test adapters deprecation notice added. NUnit will still deliver a vsix for this version.
- 461 Publish symbols with the adapter for debugging into the adapter and nunit, see
- 47 Adapter runs explicit tests when TFS TestCaseFilter is used
- 296 Mono.Cecil causes OutOfMemoryException with new .csproj PDBs
- 310 Consistent Category Display in Test Explorer Window
- 365 An exception occurred while invoking executor: Could not load file or assembly System.Runtime.InteropServices.RuntimeInformation (Thanks to halex2005 for PR 418)
- 419 Test result is Skipped when an exception has been thrown and the only warning is in TearDown
- 444 Dump file is not created when the test crashes
- 452 Adapter does not seem to respect any TestCategory filtering
- 460 Failure to load dependency assembly causes hang
Also see Release Readiness Review
Special thanks to : Joseph Musser for awesome work on this release,
and to Loren Halvorsen for the workaround for VSTest issue 261.
This version supports Visual Studio from version 2012, Update 1 and upwards, and all subsequent versions in 2013, 2015 and 2017. Visual Studio 2012 RTM is then no longer supported.
- 390 Pass DisableAppDomain and DisableParallelization settings to the Engine
- 392 Improve performance of discovery (Thanks to Navin (Microsoft) for PR 393, PR 406 )
- 394 The adapter tests are updated to use NUnit 3.8.1
- 372 netcoreapp + CultureInfo.CurrentCulture = bad time (Thanks to Aaron Housh (Dispersia) for PR 380 )
- 386 DateTime.Parse issue during test discovery with certain cultures (Also fixed by PR 380 )
- 302 BadImageFormatException building solution with unmanaged projects
- The adapter now support running .net core projects. See this post for details. Note: Only supported by the nuget adapter, not the vsix.
- The adapter now uses version 3.7 of the engine to run tests (360)
- Attachments can be added to tests (358)
- Prepared for new upcoming Test Explorer functionality, as documented in this RFC (351)
-
298
$RANDOM_SEED$ is appearing in non-test project build output - 259 An exception occurred while invoking executor 'executor://nunit3testexecutor/'
- 314 Any TestFixture deriving from a base-class which defines unit-tests will fail when the base-class is from another class-library
- 231 Improved message on failed loading. See also PR 309
- 338 Unhandled Exception when running through vstest.console.exe /listtests
- The adapter now uses version 3.6 of the engine to run tests
- Warning messages are displayed. Note that Visual Studio currently lists them as skipped.
- Multiple assertion failures in a Multiple Assert block are displayed.
- 218 No tests found to run when TestFixture is a nested class
- 256 Rename
$RANDOM_SEED$ - 258 Modify adapter to display multiple assert information
- 268 Make the Icon Larger
- 272 URL for "More information" should point to correct landing page
- 273 Report Warnings in VS adapter
- 276 Adapter requires test project pbd's to be generated for tests to be executed
- 288 Test parameters containing semicolons are truncated
Note: This was a hotfix release of the vsix package only, fixing an issue that prevented it from installing under VS2012 and 2013.
- 260 VSIX no longer visible in Visual Studio 2012
- Adds support for Visual Studio "15"
- 253 Warnings about
$RANDOM_SEED$ during build - 262 Support for VS 15
- The adapter now uses version 3.5 of the NUnit engine
- The NuGet package is now installed as a tool and no longer creates unnecessary references in the project
- 97 Tests with dynamic/random parameters are never run closed:done is:bug pri:normal
- 204 If a test writes to Console.Error, the test passes but the session fails
- 220 Visual Studio Test Adapter - Writing to test output throws an error
- 221 Change adapter package to use tools directory
- 236 Update adapter to use Version 3.5 of the Engine
- 238 Just warn upon failing to restore random seed
- 239 Remove Error-level log messages from adapter where we don't want run to fail
- 243 NuGet package and copy local
- We now use Cake to build and package the adapter.
- 198 Create Cake script for build
- 202 NUnit3 Test Adapter not running all tests in Visual Studio 2015
- 205 Adapter fails to find Mono.Cecil when targeting .NET 3.5
- The adapter now uses the NUnit 3.4.1 TestEngine.
- TestRunParameters may now be provided in the
.runsettingsfile. - Immediate text output from tests now displays in the Output window. This includes any output produced through Console.Error, TestContext.Error or TestContext.Progress.
- 132 Print to console not shown with v3 of adapter
- 138 Cannot run navigation tests under the console runner
- 145 Implement TestRunParameters inside .runsettings for runtime parameters
- 180 Upgrade to NUnit 3.4.1
- 181 Can't run requiring a 32-bit process
- 183 Use Mono.Cecil to retrieve source code locations
- 190 NUnit30Settings.xml is used by other process leads to hidden tests
- 192 Corrupt NUnit3Settings.xml causes crash
- The adapter now uses the NUnit 3.2.1 TestEngine.
- 131 NUnit test adapter not running all tests - only on VS2015
- 135 Upgrade TestEngine to 3.2.1
- 162 Visual Studio 15 support
- 163 No source location when inheriting test methods from base test fixture
- 174 Clarify that NUnit v2 tests are not discovered by v3 adapter
- 176 More Information link in vsix broken
-
This is the final production release of the 3.0 adapter. It continues to use the 3.0.1 release of the NUnit TestEngine.
-
The adapter now uses a
.runsettingsfile for all optional settings. Registry entries used in the CTP releases are no longer used.
- 49 Need a way to specify test settings
- 52 Having ApartmentAttribute on both classes and methods causes test runner to hang
- 85 Failure to run tests under vstest.console from VS2015
- 92 Provide option to run in parallel for parallelized tests
- 120 The ability to set the LevelOfParallelism attribute through the VS adapter
- 153 TFS Filter that matches no names runs all tests
- The adapter continues to use the 3.0.1 release of the NUnit TestEngine.
- 2 CI Build
- 34 Identifying Non-Primitive Parameterized Inputs in Adapter vs Console
- 50 NuGet version install script doesn't work with VS 2015
- 66 Build the adapter in AppVeyor
- 84 CopyLocal=False is an issue in a specific use case
- 94 More Information Link in Adapter Broken
- 96 Working directory is set to VS TestWindow extension directory
- 102 Package VS2012 assemblies as a private NuGet Package
- 104 Can't overload async/await methods with NUnit Test Adapter
- 106 Explicit tests appear as warnings in NUnit 3.0
- 109 NUnit 2 tests are detected as errors
- 112 Test adapter fails to load an assembly that references a class from NUnit.Framework but contains no tests
- 117 Version 3.0.8.0 as NuGet package only. No tests detected in Visual Studio 2015
- 118 Corrupted ignore.addins file in installation
- The adapter now uses the 3.0.1 release of the NUnit TestEngine.
- 81 Cannot run tests with '>' in name
- 86 Generic Test Fixtures are not getting triggered
- 88 Upgrade adapter to use NUnit 3.0.1
- The adapter now uses the released NUnit 3.0 TestEngine.
- 75 Update adapter to use final release of NUnit 3.0
- This release continues to use the NUnit RC 2 Engine.
- 14 NUnit Adapter throws System.Reflection.TargetInvocationException, even if the solution build is OK
- 56 Exception System.Reflection.TargetInvocationException after NUnit 3.0.0-beta-5 upgrade
- 68 NUnit3TestExecutor.MakeTestFilter does not create valid xml
- 69 NUnit 3.0.0-rc-2 : System.Reflection.TargetInvocationException
- 70 NUnit3TestExecutor.MakeTestFilter creates element not handled by NUnit.Framework.Internal.TestFilter
- This release uses the NUnit RC 2 Engine.
- 27 Async void methods do not show up as not runnable
- 43 Remove Wrappers for Engine Classes
- 45 Remove workaround for tests not sending events
- 53 Replace core engine
- 57 Confusing message when an NUnit V2 test is detected
-
This release continues to use the NUnit 3.0 beta-2 engine but is nevertheless able to run tests that reference NUnit 3.0 beta-3 framework.
-
When a debugger is attached, only a single worker thread is used to run tests.
-
The adapter now compensates for the fact that NUnit does not send results for tests that were skipped due to an attribute on the fixture by generating those results itself.
- 16 Adapter does not detect C++/CLI assemblies
- 26 Ignored test case does not show up as ignored
- 33 Inconsistent display behavior in Test Explorer
- 36 Option to set number of worker threads
This release was issued to correct a problem with locking of assemblies in ctp-2.
- 29 Latest test adapter locking dlls
-
The adapter now uses the new nunit.core.engine to load and run tests, eliminating ad-hoc code that worked directly with the framework. This will allow us to much greater flexibility in the future.
-
The adapter has been upgraded to use the beta-2 release of the NUnit core engine. Because the API has changed from beta-1, the adapter can only run tests built against the beta-2 release of NUnit.
- 13 Adapter will not load as a NuGet package
- 17 Can't read app.config settings within test methods
- 18 Separate NUnit3TestDemo from NUnitTestAdapter solution
- 19 Use core engine
- 20 Upgrade NUnit to beta-2
- Initial release of the test adapter using NUnit 3.0. Note that the adapter may not be used to run tests written against earlier versions of NUnit. The original adapter is still available for that purpose and both adapters may be installed if necessary.
- Tested for up to VS2015 Pre-release CTP 6
- Updated to use NUnit 2.6.4
- Adapter does not try to discover tests if the nunit.framework version is 3.0 or greater
- #61 Confusing NUnit version message made clearer
- #62 Adapter uses shadowcopy setting in discoverer but not in the executor
- Tested for up to VS2013 Update 3
- Bugs 39 and 40 was inability to run under VS2012. This is now fixed.
- #24 Long-running tests ignored
- #34 Adapter causes ArgumentException to be thrown by Microsoft logger proxy's SendMessage method
- #37 TestExecution throws Exception System.InvalidOperationException in TFS Build
- #38 NUnit only accepts absolute paths to test assembly
- #39 VSTest unable to find NUnit tests since 1.1.0.0
- #40 NUnit version 1.1.0.0 is broken with test class which ran under 1.0.0.0
- Support for NUnit 2.6.3
- Tested for up to VS2013 Update 2 RC
- Shadow copy now disabled by default, see issue #7 Unable to disable shadow copy.
- Registry settings added for some customization options, see Tips and Tricks
- All code moved to github
- #13 Category attribute not working with TFS test case filter
- #21 Xamarin.iOS NUnit project causes adapter to throw
- This is the release version 1.0 of the test adapter.
- #1208148 The test result output node is not shown for debug/trace statements
- This is the release candidate for version 1.0 of the test adapter.
- #1208161 NUnit Test Adapter runs [Explicit] unit tests in TFS Build
- #1210536 No Source Available for Async Tests
- #1165188 Clicking "Run Selected Tests" doesn't show Trace.WriteLine() output
- Support for Visual Studio 2013 Preview
- #1189268 Profile a test will crash with exception
- #1188000, adapter dont work with solutions with only .net 2.0/3.5 project
- Hotfix for debug issue
- #1174925 Add support for installing the adapter from NuGet
- #1155617 Grouping by class name in VS 2012 doesn't work
- #1165359 Exception after building Coded UI test
- #1116747 vstest.executionengine.x86.exe does not terminate
- #1093178 Eliminate unnecessary files from VSIX
- Works with Visual Studio 2012 Update 1 as well as the RTM.
- Supports filtering and sorting tests by Traits under Update 1.
- Supports use of standard filter expressions when running under TFS Update 1.
- NUnit Categories specified on the fixture class are now recognized and honored.
- 1074891 Can't test multiple assemblies referencing different NUnit versions
- 1075893 Test execution fails if solution contains native C++ project
- 1076012 No source information found for async test methods
- 1087629 TestFixture Category not being recognised as traits in VS2012 update 1
- 1091020 Adapter doesn't support TFS Build traits/test case filtering
- 1074544 Failures in Test Discovery not reporting sufficient information
- 1072150 NUnit adapter 0.93 won't run selected tests
- Works with Visual Studio 2012 RTM. Some features require the November CTP update.
- The adapter now uses NUnit 2.6.2. Among other things, this allows us to support async test methods. See the NUnit Release Notes for more info.
- Source file and line number can now be found for test cases that have an alternate name set.
- Console output from tests is now displayed in the Visual Studio Output window.
- TestFixtureSetUp and TestFixtureTearDown errors are now displayed in the Output window.
- The caret line (------^) is no longer displayed in the IDE since it depends on use of a fixed font.
- Tests may now be grouped and filtered by Category (only under the November CTP update for VS2012).
- 1021144 Text output from tests not displayed in Visual Studio IDE
- 1033623 Not possible to include or exclude tests based on [Category] attribute Released
- 1040779 Null reference exception on generic test fixtures
- 1064620 Support async test methods
- 1065209 Should call both RecordEnd and RecordResult at end of a test
- 1065212 Upgrade NUnit to 2.6.2
- 1065223 Error messages assume a fixed font, but don't get one
- 1065225 No display for TestFixtureSetUp/TearDown or SetUpFixture errors
- 1065254 Cannot open test from Test Explorer for tests in a Parameterized Test Fixture
- 1065306 Generic Fixtures aren't discovered.
- 1066393 Unable to display source for test cases with an alternate name set
- 1066518 Executed fast test appears in Not Run category in Test Explorer
- Works with Visual Studio 2012 Release Candidate
- Uses NUnit 2.6
- 992837 Unable to Debug using VS Test Adapter
- 994146 Can't run tests under .NET 2.0/3.5
- Built against Visual Studio 11 Beta 1
- Uses NUnit 2.6
- First release of the test adapter. Compatible with the Visual Studio 11 Developer Preview.
- Uses NUnit 2.6.
Copyright (c) 2018 The NUnit Project - Licensed under CC BY-NC-SA 4.0