Allow specifying Test Selection Language filters in VSTest runsettings

[mletterle]

This allows using dotnet test as a drop in replacement for a
nunit-console run with the --where option.

i.e.:

nunit-console SomeTests.dll --where "cat == SomeCategory"
becomes
dotnet test SomeTest.csproj -- NUnit.TestsWhere="cat == SomeCategory"

Related: #425, #655

[OsirisTerje]

@mletterle The idea of adding test filters to the runsettings is a good one, I like that one!
But, I don't really understand the connection with dotnet test, which already supports the --filter. Can you elaborate a bit on this?

Also, is this really two different things in one PR?

Do you see this PR as resolving issues with #425 and #655

[mletterle]

@mletterle The idea of adding test filters to the runsettings is a good one, I like that one!
But, I don't really understand the connection with dotnet test, which already supports the --filter. Can you elaborate a bit on this?

Sure, the filter used by --filter is VSTest's filter dialect which is different then NUnit's native Test Selection Language

Not only is the syntax different, but as far as I can tell VSTest's filtering is far more limited than NUnit's native filtering.

This PR allows one to use NUnit's native Test Selection Language from the dotnet test command (and also via dotnet vstest)

Also, is this really two different things in one PR?

Do you see this PR as resolving issues with #425 and #655

The two issues have a common source, they're both about selecting which tests you actually want to run under NUnit.

#425 is related because this allows you to run dotnet test -- NUnit.TestsWhere="arbitrary Test Selection Language Filter"

#655 is related because, as far as I can tell it's actually not possible to use VSTest filter to select a specific parameterized test, but it is possible with the Test Selection Language, so for example you could do:

dotnet test -- NUnit.TestsWhere="test == 'My.Name.Space.TestFixture.TestMethod(5)'"

[OsirisTerje]

Thanks @mletterle ! Makes sense!

I'll add some review comments :-)

[mletterle]

@OsirisTerje Is there any further changes you'd like to see?

[OsirisTerje]

I think this is fine. I would like to have this out together with the other changes in 3.16. Have you merged the latest changes from master in here? And verified there are no crashes there ?

[mletterle]

I rebased the PR against current master, everything appears ok.

[OsirisTerje]

Btw, are you writing blog posts? This change could deserve a post.

[mletterle]

It's been a long time since I wrote a blog post ^_^

It's not a very complicated feature, I could try and put something together this weekend, if it would be helpful.

[OsirisTerje]

It's obvious to those close to the issue at hand :-) But not so for many others. I am often a bit surprised by how well received simple explanations can be.
So it would be awesome if you did! I'll refer to it in the release notes afterward too.

[mletterle]

@OsirisTerje Blog posted: http://blog.prokrams.com/2019/12/16/nunit3-filter-dotnet/

[OsirisTerje]

Awesome! Thanks a lot!

[OsirisTerje]

@mletterle Version 3.16 has been published. Links to your blogpost added in release notes and on the Trick and Trips page. :-)

[goblinmaks]

@mletterle Is it possible to pass string parameters as part of filter value like
dotnet test -- NUnit.TestsWhere="test == 'My.Name.Space.TestFixture.TestMethod("myString")'"
?

[OsirisTerje]

@goblinmaks Something like that should work, perhaps with some escaping. If you also set Verbosity to 5, you should see all settings being listed , can be good for checking what the adapter resolves it into :-)

[mletterle]

Yeah, shell escaping shenanigans comes into play here, it might be easier to just define the filter in a .runsettings file