concurrent runs screw up log report

[CendioOssman]

Testing multiple browsers at the same time confuses the reporting somehow and you can't really see which instance failed or succeeded, or where the output belongs.

Examples:

https://travis-ci.org/CendioOssman/noVNC/jobs/179805509
https://travis-ci.org/CendioOssman/noVNC/jobs/179770987

[4kochi]

I am not sure if the output is from the reporter. Actually it supports concurrent browsers. Could it be that the output is from some console.log() statements?

[CendioOssman]

Yes, it is. However I would expect that output to be either at the start or bottom, or at the test that generates the output. Right now it is very random.

More concerning is that:

a) there are tests that are marked as failed (red, and x), yet the entire run is considered a success
b) you cannot see the test results for each platform. So failures might just be for one platform, and a success might be skipped for others.

[4kochi]

Regarding the console.log this seems to be the same problem as in #78. There is also an open issue in the karma repo (karma-runner/karma#2342) which means this could be a general in how the reporters are supposed to use the logging API of karma. I will keep an eye on this

[4kochi]

The other problem with the different platforms is hard to track down for me. Might a problem which only occurs when using custom saucelab launchers. I will try to reproduce it.

[wesleycho]

This is a problem in karma itself IMO - karma is coupling the defining of default reporters with logging wired in, so reporters that are using the base reporter decorator are logging by default, which is probably not the intention. I have a proposed solution (karma-runner/karma#2342 (comment)) that I am leaning towards, and it would avoid needing to change anything in this library itself.

[4kochi]

Thanks for the information @wesleycho