Team Foundation Build support

[agoodwin]

Is there any way to get this test adapter to run tests on a build server?

I tried the basic stuff of checking the files from the VSIX package into source control (in the location where the build controller looks for custom test adapters), then setting up a test source in the build definition to look for the KarmaTestAdpater.json file. I also tried a trick I've seen elsewhere on the internet, which is to make sure all of the test adapter DLLs have a *.TestAdapter.DLL filename.

Unfortunately the build just doesn't run the Karma tests, and there doesn't seem to be any logging or documentation I can find to troubleshoot this. There are no errors or warnings that I can find, the build just quietly runs no tests.

With the same approach we have had some success with incorporating other JavaScript test adapters into our build, although we really like using this test adapter for running our Karma tests in Visual Studio and it would be great it we could get it to work in our build as well.

I can give a bit more information about things I've tried if there is someone who can help with this,

Thanks

[zacharybussey]

We integrated Karma into our build by just running the karma command on the commandline from PSExec. I didn't see any clear way of integrating this plugin in the build process.

Also, by separating this out, we are running the tests on a different box than the build machine, so that, potentially, we can run against different versions of IE on different boxes. Just haven't gotten around to setting that up yet.

[agoodwin]

Thanks for the reply. Am I right in saying that by using the command line your build either passes or fails – i.e. there is nothing in Visual Studio's build summary page about which Karma tests in particular failed or passed?

[zacharybussey]

Ah, so, our setup is a little more complicated than I led on.

• In our build script, we first copy all of the JS files and the karma.config file over to a different JS testing box.
• Then, we call PSExec to run Karma on the other box, and record the result code.
• When Karma is done, it creates a result html file using the dhtml reporter.
• This html page with the results gets copied back to the build box and put in the build drop folder. We also have other result files from other testing libs in the same folder.
• Then, if the result code we got back is 1, we fail the build.

So, no, we don't really show anything in the results screen, other than the fact that the JS tests failed. But, we have the same system in place for our other testing frameworks, so everyone knows to open up the drop folder and see what failed already.

We've been using that setup for a few months without much issue. Firefox sometimes does not start up in time, probably because it is auto updating. So, we sometimes get failures from that. Also, Chrome and PSExec do not work well together, so, we aren't running any Chrome tests right now. I'd like to get some of that ironed out, but, other projects are more important right now.

[agoodwin]

Thanks for the clarification. To be honest we're probably going to have to go with something like that ourselves, I was just really hoping someone might have already got this test adapter working in a TFS build.