methodformat always includes namespace and class name regardless of setting

[moly]

I'm using junit.testlogger 2.1.78 and dotnet 3.1.402.

I have a test class called LayerConverterTest in a directory test/MyProject.Test/Json. It's namespace is MyProject.Test.Json

If I run dotnet test --no-build --logger:"junit;LogFilePath=..\artifacts\{assembly}-test-result.xml;MethodFormat=Default;FailureBodyFormat=Verbose"

The resulting report wrongly includes the full namespace and class in the name attribute.

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
  <testsuite name="MyProject.Test.dll" tests="4" skipped="0" failures="0" errors="0" time="0.1136953" timestamp="2020-10-01T11:23:36" hostname="executor://xunit/VsTestRunner2/netcoreapp" id="0" package="MyProject.Test.dll">
    <properties />
    <testcase classname="MyProject.Test.Json.LayerConverterTest" name="MyProject.Test.Json.LayerConverterTest.CanConvert_LayerSubclass_True(type: typeof(MyProject.Json.Layers.Image))" time="0.0070400" />
    <testcase classname="MyProject.Test.Json.LayerConverterTest" name="MyProject.Test.Json.LayerConverterTest.CanConvert_LayerSubclass_True(type: typeof(MyProject.Json.Layers.PreComp))" time="0.0000645" />
    <testcase classname="MyProject.Test.Json.DataTest" name="MyProject.Test.Json.DataTest.Deserialise_ValidJson_PropsPopulated" time="0.0544729" />
    <testcase classname="MyProject.Test.Json.LayerConverterTest" name="MyProject.Test.Json.LayerConverterTest.Read_ImageJson_ReturnImageObject" time="0.0521179" />
    <system-out>Junit Logger does not log standard output</system-out>
    <system-err>Junit Logger does not log error output</system-err>
  </testsuite>
</testsuites>

If I change the method format and run dotnet test --no-build --logger:"junit;LogFilePath=..\artifacts\{assembly}-test-result.xml;MethodFormat=Full;FailureBodyFormat=Verbose"

The resulting report includes the namespace and class name twice.

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
  <testsuite name="MyProject.Test.dll" tests="4" skipped="0" failures="0" errors="0" time="0.0837441" timestamp="2020-10-01T11:46:24" hostname="executor://xunit/VsTestRunner2/netcoreapp" id="0" package="MyProject.Test.dll">
    <properties />
    <testcase classname="MyProject.Test.Json.LayerConverterTest" name="MyProject.Test.Json.LayerConverterTest.MyProject.Test.Json.LayerConverterTest.CanConvert_LayerSubclass_True(type: typeof(MyProject.Json.Layers.Image))" time="0.0107667" />
    <testcase classname="MyProject.Test.Json.LayerConverterTest" name="MyProject.Test.Json.LayerConverterTest.MyProject.Test.Json.LayerConverterTest.CanConvert_LayerSubclass_True(type: typeof(MyProject.Json.Layers.PreComp))" time="0.0000384" />
    <testcase classname="MyProject.Test.Json.DataTest" name="MyProject.Test.Json.DataTest.MyProject.Test.Json.DataTest.Deserialise_ValidJson_PropsPopulated" time="0.0395664" />
    <testcase classname="MyProject.Test.Json.LayerConverterTest" name="MyProject.Test.Json.LayerConverterTest.MyProject.Test.Json.LayerConverterTest.Read_ImageJson_ReturnImageObject" time="0.0333726" />
    <system-out>Junit Logger does not log standard output</system-out>
    <system-err>Junit Logger does not log error output</system-err>
  </testsuite>
</testsuites>

[Siphonophora]

@moly I couldn't repo this so far and I probably need more detail.

First, could you confirm you are on 2.1.78. <system-out>Junit Logger does not log standard output</system-out> should be a prior version.

Assuming that doesn't help, I think a csproj file and a test class would help a lot. I'm wondering if the reason I couldn't repo this is due to your exact framework / dependencies. Also, I haven't seen a test name rendered like MyProject.Test.Json.LayerConverterTest.CanConvert_LayerSubclass_True(type: typeof(MyProject.Json.Layers.Image)) before, so I want to ensure I understand how that was generated.

Also, could you confirm you aren't getting any warnings from the logger in the console output for your tests about being unable to parse names?

Thanks

[moly]

@Siphonophora
I've updated to 2.1.78 and still get the same issue. I've pushed a sample project here: https://github.com/moly/junitrepro

The project was created with the latest dotnet using dotnet new xunit. There aren't any dependencies other than xunit and the junit logger.

If I run dotnet test --logger:"junit;LogFilePath=..\artifacts\{assembly}-test-result.xml;MethodFormat=Full;FailureBodyFormat=Verbose"

I get:

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
  <testsuite name="sampleproject.test.dll" tests="1" skipped="0" failures="0" errors="0" time="0.0021342" timestamp="2020-10-08T09:22:09" hostname="UKONG-NB0022" id="0" package="sampleproject.test.dll">
    <properties />
    <testcase classname="sampleproject.test.json.layers.UnitTest1" name="sampleproject.test.json.layers.UnitTest1.sampleproject.test.json.layers.UnitTest1.Test1" time="0.0021342" />
    <system-out>[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.0 (64-bit .NET Core 3.1.8)
[xUnit.net 00:00:00.64]   Discovering: sampleproject.test
[xUnit.net 00:00:00.68]   Discovered:  sampleproject.test
[xUnit.net 00:00:00.68]   Starting:    sampleproject.test
[xUnit.net 00:00:00.76]   Finished:    sampleproject.test
</system-out>
    <system-err></system-err>
  </testsuite>
</testsuites>

The full namespace and class name is duplicated in the name attribute.

[Siphonophora]

@moly Great. I will take a look.

[IshakAtLEGO]

Any updates on this?

[Siphonophora]

Hi, were working on the v3 release now, and I will take a look at this as part of that release

[CoskunSunali]

Currently struggling with the same issue. I'm on version 3.0.98

[Siphonophora]

@moly, @CoskunSunali, @IshakAtLEGO

Are all of you using xUnit? The repro that moly provided, did give an example of MethodFormat not working correctly.

On mstest we are getting the expected behavior:

<!-- MethodFormat=Default-->
    <testcase classname="JUnit.Xml.TestLogger.NetFull.Tests.FailingOneTimeSetUp"
                    name="TestA" time="0.0000001">

<!-- MethodFormat=Class-->
    <testcase classname="JUnit.Xml.TestLogger.NetFull.Tests.FailingOneTimeSetUp" 
                     name="FailingOneTimeSetUp.TestA" time="0.0000001">

<!-- MethodFormat=Full-->
    <testcase classname="JUnit.Xml.TestLogger.NetFull.Tests.FailingOneTimeSetUp" 
                    name="JUnit.Xml.TestLogger.NetFull.Tests.FailingOneTimeSetUp.TestA" time="0.0000001">

[CoskunSunali]

@Siphonophora I’m using xUnit.

[ishakf]

I am also using xUnit.

[Siphonophora]

@CoskunSunali @ishakf

We have a fix for this published on MyGet https://www.myget.org/feed/spekt/package/nuget/JunitXml.TestLogger/3.0.102 If you get a chance to test it, let me know if this resolves your issue.

[IshakAtLEGO]

This has fixed the issue. I assume that this will be available on Nuget as well?

[Siphonophora]

Great.

Yes it will. I have a fix for one other issue that's about to get merged, and we will release both together. I will comment here when its on NuGet

[Siphonophora]

@moly this is on nuget now in 3.0.110.

[tamaramironov]

Hello!

I am wondering if it was possible to add the old, "buggy" option back now😁

We've been publishing test reports generated by junit.testlogger to our requirements management software (RMS). The integration was based on test case name and is now broken with the latest upgrade. How it's used to work:

using Microsoft.VisualStudio.TestTools.UnitTesting;

[TestMethod("@RMS_TEST_NUMBER RMS Test name")]
public async Task MethodUnderTest_Scenario_Result()
{
    Assert.Whatever();
}

Testlogger used to produce the following result that was parsed by our RMS
<testcase classname="Whatever" name="@RMS_TEST_NUMBER RMS Test name" time="0.1"/>

I guess VS unit testing was publishing the name from TestMethod attribute. And now naturally it's this, and we can't link it to RMS anymore
<testcase classname="Whatever" name="MethodUnderTest_Scenario_Result" time="0.1"/>

So, we'd really appreciate if we could publish TestResultInfo.Name again, as a testcase name or anyhow, can this be considered for the future versions @Siphonophora?

Thank you!

[Hackmodford]

Update fixed the issue for me. I am using xunit as well.

[CoskunSunali]

I am wondering if it was possible to add the old, "buggy" option back now😁

@tamaratomilova Cannot you just use the MethodFormat=Full option?

[tamaramironov]

I am wondering if it was possible to add the old, "buggy" option back now😁

@tamaratomilova Cannot you just use the MethodFormat=Full option?

Nope. As per readme, MethodFormat Full will add the assembly/namespace/class to the test method name, while I need the test method name to be populated from TestMethod attribute instead of the actual method name. Different thing.