NUnit3TestAdapter 3.15 OneTimeSetUp not working anymore

[f-bourqui]

Visual Studio 2017 v15.9.14

The first time I run a test, OneTimeSetUp is called, then every other runs fails. I had no problems with 3.14. I made a new project to reproduced it with minimal code and I can run multiple time my test and it always succeed with 3.14, but as soon as I upgrade to 3.15, it is never called anymore.. If I restart VS, it work only once and all following try fails. Can be reproduced with nuget package or VS extension.

using NUnit.Framework;

namespace NunitAdapter315Bug
{
    [SetUpFixture]
    public class ProjectSetup
    {
        public static bool oneTimePassed = false;

        [OneTimeSetUp]
        public void OneSetup()
        {
            oneTimePassed = true;
        }
    }
}

using NUnit.Framework;

namespace NunitAdapter315Bug
{
    [TestFixture]
    public class TestNunitAdpaterTest
    {
        bool setupPassed = false;
       
        [SetUp]
        public void Setup()
        {
            setupPassed = true;
        }

        [Test]
        public void TestIt()
        {
            Assert.IsTrue(ProjectSetup.oneTimePassed);
            Assert.IsTrue(setupPassed);
        }
    }
}

[MattKeenum]

I am able to reproduce this issue with the above code. However, if I have the TestNunitAdapterTest class inherit from ProjectSetup, everything works as expected. Not sure if this is intended behavior or not.

namespace NUnitTest
{
    [SetUpFixture]
    public class ProjectSetup
    {
        public static bool oneTimePassed = false;

        [OneTimeSetUp]
        public void OneSetup()
        {
            oneTimePassed = true;
        }
    }

    [TestFixture]
    public class TestNunitAdpaterTest : ProjectSetup
    {
        bool setupPassed = false;

        [SetUp]
        public void Setup()
        {
            setupPassed = true;
        }

        [Test]
        public void TestIt()
        {
            Assert.IsTrue(oneTimePassed);
            Assert.IsTrue(setupPassed);
        }
    }
}

[mikkelbu]

I'm quite sure that this is the same problem as in nunit/nunit#3356 (see the comments for more info), but I've not had time to look into how to fix it.

[ggeurts]

I have also had to revert to version 3.14 because tests in .NET Core started failing due to not running of SetUpFixture with OneTimeSetup method.

[jhbell]

If anyone else is having trouble downgrading to version 3.14 to avoid this bug, here are the steps I took:

1. Uninstall NUnit3TestAdapter from Visual Studio and make sure auto-update for your extensions are turned off.
2. Download the NUnit3TestAdapter-3.14.0.vsix file from the releases page here. (Under Assets)
3. Double click the .vsix file which will launch the VSIX Installer and install 3.14 on your desired version of visual studio.

[OsirisTerje]

Thanks @jhbell !

Will try to get out a fix within this week, probably using a feature flag to enable/disable the pre-filtering.

[viceice]

@jhbell Your solution only works for the vsix, we don't use that. We have to downgrade the NUnit3TestAdapter package reference to 3.14.0

[OsirisTerje]

@viceice Downgrading a nuget package is much more simple than the vsix, since all you have to do is either : 1) Use the Tools/Manage nugetpackages in Visual Studio and select the 3.14 version 2) If you have a SDK based project, just edit the csproj file directly. :-)

However, the fix is on its way now, a PR is up, waiting for a quick review. Hopefully we can get it out tomorrow, Aug 30.

[OsirisTerje]

@SmartLibertyFBO @ggeurts @viceice There is now a beta version of the fix in https://www.myget.org/feed/nunit/package/nuget/NUnit3TestAdapter/3.15.1-dev-01134 . Would appreciate if you checked it.

[viceice]

I can test it earliest on monday

[OsirisTerje]

@SmartLibertyFBO 3.15.1 hotfix released now.

[f-bourqui]

It seems to work with my test solution, thanks

[viceice]

Works here