Visual Studio Test Task fails on TFS 2017 RTM during build

[srivatsamarichi]

Hi,

I am running Unit Tests as part of my build in TFS 2017 RTM. I am using VS Test task to do that. When i pass Test Filter Criteria as "TestCategory=UnitTests", the test fails. But it would have identified the tests and shown in the Test Summary post build. PFA logs for the same. Request any help here.

Thanks,
Srivatsa
5_Test Assemblies.txt
6_Test Assemblies(XUnit and NUnit).txt

[acesiddhu]

@srivatsamarichi looks like you are using nunit tests. Below error from logs suggest that this is not a valid filter that nunit understands. Please use nunit filters instead.

"Exception filtering tests: No tests matched the filter because it contains one or more properties that are not valid (Category). Specify filter expression containing valid properties (DisplayName, FullyQualifiedName) and try again."

[srivatsamarichi]

@acesiddhu I have passed the filters in my tests as "TestCategory=UnitTests" for MSTests. It just works fine. I have couple of Nunit and Xunit tests as well. I have passed the test filter for each test method as [Category=Nunit]. But still the tests are not getting picked up during my build. If i don't specify any filter criteria, then it considers all the tests including my selenium scripts and will give me the summary. I don't want to run selenium tests as part of my builds and hence i am trying out to filter out here. Could you tell what are those filters?

Thanks,
Srivatsa

[acesiddhu]

can you ping me the test method body here for both kind of tests. I want to see attributes in the test method

[srivatsamarichi]

For NUnit:

using System.Web.Mvc;
using FabrikamFiber.DAL.Data;
using FabrikamFiber.Web.Controllers;
using NUnit.Framework;
using NUnit.Mocks;
using Xunit;
using Xunit.Extensions;
using Microsoft.VisualStudio.TestTools.UnitTesting;

[TestFixture]
public class HomeControllerTest
{
    [Xunit.Fact]
    [Category("UnitTests")]
    public void IndexReturnsNonNullView()
    {
        var serviceTicketRepo = new DynamicMock(typeof(IServiceTicketRepository));
        var messageRepo = new DynamicMock(typeof(IMessageRepository));
        var alertRepo = new DynamicMock(typeof(IAlertRepository));
        var scheduleItemRepo = new DynamicMock(typeof(IScheduleItemRepository));

        var controller = new HomeController(
            serviceTicketRepo.MockInstance as IServiceTicketRepository,
            messageRepo.MockInstance as IMessageRepository,
            alertRepo.MockInstance as IAlertRepository,
            scheduleItemRepo.MockInstance as IScheduleItemRepository
        );

        var result = (ViewResult)controller.Index();
    }

---

For MSTest

using System;
using System.Linq;
using System.Linq.Expressions;
using System.Web.Mvc;
using FabrikamFiber.DAL.Data;
using FabrikamFiber.DAL.Models;
using FabrikamFiber.Web.Controllers;
using Microsoft.VisualStudio.TestTools.UnitTesting;

[TestClass()]
public class CustomersControllerTest
{
    MockCustomerRepository mockCustomerRepo;
    CustomersController controller;

    [TestInitialize()]
    public void SetupController()
    {
        mockCustomerRepo = new MockCustomerRepository();
        controller = new CustomersController(mockCustomerRepo);
    }
    
    [TestCategory ("UnitTests"),TestMethod()]
    public void CreateInsertsCustomerAndSaves()
    {
        controller.Create(new Customer());

        Assert.IsTrue(mockCustomerRepo.IsInsertOrUpdateCalled);
        Assert.IsTrue(mockCustomerRepo.IsSaveCalled);
    }

    [TestCategory("UnitTests"),TestMethod()]
    //[ExpectedException(typeof(ArgumentNullException))]
    public void CreateNullCustomer()
    {
        controller.Create(null);
        Assert.IsTrue(mockCustomerRepo.IsSaveCalled);

    }

Thanks,
Srivatsa

[srivatsamarichi]

PFA scripts for the same.

CustomersControllerTest.txt
HomeControllerTest.txt
ServiceTicketsControllerTest.txt

Thanks,
Srivatsa

[srivatsamarichi]

Any updates here?

[acesiddhu]

@srivatsamarichi we will take a look at this today. Will keep you posted

[srivatsamarichi]

@acesiddhu Thanks for the update.

[acesiddhu]

@tanvi-soni can you check this out? Thanks

[acesiddhu]

@srivatsamarichi for nunit test you need to put this filter

TestCategory=UnitTests
This will be valid if the test has attribute [Category("UnitTests")]

Can you try with this property name.

[srivatsamarichi]

@acesiddhu I have passed the same arguments and filters. But it's still failing.

[acesiddhu]

is this the test case?. This is a mix of nunit and xunit. is this supposed to be nunit or xunit? I tried with nunit test case
[TestFixture]
public class HomeControllerTest
{
[Xunit.Fact]
[Category("UnitTests")]
public void IndexReturnsNonNullView()
{
}
}

[srivatsamarichi]

@acesiddhu Yep this is the one which i am trying for. This is a mixture of both NUnit and XUnit. I have 14 tests out of which 11 are identified. The other 3 are the ones which are not getting identified.

[acesiddhu]

this particular test method is a xunit test and not nunit. is this test case getting identified? I doubt because this test is treated as xunit test and not nunit (as Fact is the attribute)

[srivatsamarichi]

If i don't specify any filters in test step then by default it will detect all the tests including my selenium and will give me the test summary. But i don't want to execute my selenium as part of my build and hence i am looking something like this. I dunno how it's getting identified when no Test Filter Criteria is passed.

[acesiddhu]

It will get identified when no test filter is given as it will run all tests. above test is a xunit test. TestCategory filter is valid for nunit only. you either

1. need to make this test nunit test
2. or add different xunit level attribute and use corresponding filers.
   See below and use this filter Category=UnitTests
   [TestFixture]
   public class HomeControllerTest
   {
   [Xunit.Fact]
   [Trait("Category", "UnitTests")]
   public void IndexReturnsNonNullView()
   {
   }
   }

[srivatsamarichi]

This indeed identified. If we could have a quick 5 minute talk would it be fine? I have something else to show you as well. Hard to explain here. [email protected] you can ping me on Skype For business. Would really help if you could :-)

[acesiddhu]

Closing the issue. You can drop a mail at devops_tools AT Microsoft DOT com for any other issues