using "dotnet test" commandline with WinUI test throws null reference exception from generated code

[ScottAtBarco]

In order to get CI/CD github actions working for a unit test gate for my WinUI app, I need to get "dotnet test" to work from the command line and ultimately from the .yml file. When I build and then run the command "dotnet test --no-build" all the tests run except those that are UI test dlls. For the UI tests, I get the following exception from the generated code:

Failed Thing_CanCreate [53 ms]
  Error Message:
   Test method NameSpace.Thing_CanCreate_generated threw exception:
System.NullReferenceException: Object reference not set to an instance of an object.
  Stack Trace:
      at NameSpace.Thing_CanCreate_generated() in C:\Path\To\WinUIEx.TestTools.MSTest.CodeGenerator\WinUIEx.TestTools.MSTest.TestGenerator\ThingTests_generatedUITests.cs:line 14

The code is something like:

public class Thing
{
     private BitmapImage _bmi; // why we must run on UI thread to test
     public Thing()
     {
         _bmi = new BitmapImage();
     }
     // other methods
}

A dirt simple test, silly as it is, that fails:

[TestClass]
public partial class ThingTests
{
    [WinUITestMethod]
    public void Thing_CanCreate()
    {
        var thing = new Thing();
        Assert.IsNotNull(thing);
    }
    // other tests
}

When I run inside VS2022, all the tests pass. Using "dotnet test", all the tests fail as described above.

Am I doing something wrong, or is this not supported? How are people incorporating UI tests like this into their github workflows?

[dotMorten]

Dotnet test isn’t supported. It is expecting to run the tests as packaged test apps which require vstest.console.

having said that, the WinUIEx test libraries are going to be deprecated as I don’t have the time to maintain this and the unit test story for WinUI has changed since this library was initially written

[ScottAtBarco]

Thanks for the quick response. That makes sense, I guess. Can you point me to anything that describes how “the unit test story for WinUI has changed”? From: Morten Nielsen ***@***.***> Sent: Wednesday, June 21, 2023 12:19 PM To: dotMorten/WinUIEx ***@***.***> Cc: Bigger(EXT), Scott ***@***.***>; Author ***@***.***> Subject: Re: [dotMorten/WinUIEx] using "dotnet test" commandline with WinUI test throws null reference exception from generated code (Discussion #127) Dotnet test isn’t supported. It is expecting to run the tests as packages test apps which require barest.console. having said that, the WinUIEx test libraries are going to be deprecated as I don’t have the time to maintain this and the unit test story for WinUI has changed since this library was initially written — Reply to this email directly, view it on GitHub<#127 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A5LLZC7WMXNZY4MFHA2INEDXMNCJDANCNFSM6AAAAAAZPEKG34>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>> This message is subject to the following terms and conditions: MAIL DISCLAIMER<https://www.barco.com/en/maildisclaimer>

[gakera]

Yes @dotMorten I'd love to know this as well, what is the unit test story for WinUI ? We've just started development on a WinUI 3 project and are looking at how testing can be automated as much as possible. We've already got core services isolated and testable "the regular way" but are looking at ways to test UI more, and I thought the WinUIEx test libraries were a good way to go?

For example, stuff that manages window position between session (don't ask about the requirements...) - In the test project I have I'm unable to just new WindowEx() because it's not "COM Registered" within that project, so any service that depends on a WindowEx object I've so far been unable to test - at least without a major overhaul.

Another case I'm trying to solve is the "Selenium" type tests of "Click a thing, verify binding, output, side-effects, etc" scenario. I've found some WinAppDriver stuff but haven't fully investigated what to do there. Not saying that the WinUIEx test library will or should help me there, just ... discussing.

I'm hesitant to create a dedicated discussion topic related to my investigation based on your response that the WinUIEx test libraries are going to be deprecated, I don't want to clutter with non-desirable discussion.