No file produced when test errors

[ianwremmel]

If I run a test in e.g. IE 11 via sauce labs and inadvertently include a backtick in my source code, the karma console will report something like

E 11.0.0 (Windows 7 0.0.0) ERROR
  Invalid character
  at /var/folders/m1/x8tv0dkd40d636mmqw7lq5_c0000gn/T/packages/test-helper-file/src/file.shim.js:93:0 <- /var/folders/m1/x8tv0dkd40d636mmqw7lq5_c0000gn/T/911df24c8515a218c68c8d72f41a3643.browserify:59086

but no report file will be produced. As a result, any CI tools that rely on structured output (read: jenkins) will consider the build a success.

[dcendents]

I have the same problem with Chrome when trying to load a json file with errors in it

[billhimmelsbach]

Same problem here. It happens only if there's something like a syntax error while Karma is initially loading the .js files, before the tests begin.

(This might be just a plain Karma problem, however, since I believe it has similar behavior even with the normal reporter when it encounters a problem while first loading the js files?)

[joeyparrish]

Just ran into this myself. The reason this is so troubling is that with --reporters junit, not only do you not get an XML file, but the error message doesn't appear on stdout or stderr, either. My build bot has no log whatsoever to tell me what went wrong, only that Karma returned a non-zero exit code.

The junit reporter should at least write this log to stderr instead of hiding it.

[joeyparrish]

The error message you'd like to see comes in through the reporter's onBrowserError method. The junit reporter does not replace this method, but it does replace the base class's message adapters:

  this.adapters = [
    function (msg) {
      allMessages.push(msg)
    }
  ]

Later, it takes those collected messages and sticks them all in the XML:

  this.onBrowserComplete = function (browser) {
    var suite = suites[browser.id]
    var result = browser.lastResult
    if (!suite || !result) {
      return // don't die if browser didn't start
    }

    suite.att('tests', result.total ? result.total : 0)
    suite.att('errors', result.disconnected || result.error ? 1 : 0)
    suite.att('failures', result.failed ? result.failed : 0)
    suite.att('time', (result.netTime || 0) / 1000)

    suite.ele('system-out').dat(allMessages.join() + '\n')
    suite.ele('system-err')

    writeXmlForBrowser(browser)

    // Release memory held by the test suite.
    suites[browser.id] = null
  }

When things fail badly, suite is null inside onBrowserComplete(), so no XML is written.

Instead, if we do this, everything works as you might hope:

  this.onBrowserComplete = function (browser) {
    var suite = suites[browser.id]
    var result = browser.lastResult
    if (!suite || !result) {
      // Make sure we report any error messages to stdout if we are not going
      // to create any XML.
      process.stdout.write(allMessages.join() + '\n');

      return // don't die if browser didn't start
    }
    // ...

PR coming soon!

[hicom150]

I cannot reproduce this error with latest version of "karma": "^3.1.4" and master of "karma-junit-reporter" 😓

@joeyparrish can we create a minimal testcase to check this issue?

[conner-fallone]

We also have an issue with the result xml file not being generated for one of our 4 test runs. It just so happens that this one test run has way more tests than the other 3, and is more complex. The results file is generated on my local no problem, but when ran as part of a build process, the results file is not generated even though the tests run.

[jcompagner]

We also have that same problem as @fallXone above
all test are run i see that in the jenkins console log. but the output file is not generated
this is really annoying because jenkins build will fail then because it can't find the file that it expect that it is generated.

Also this doesn't happen all the time..

[jcompagner]

i did create a new case here with full debug logging: #188