🚀 Feature: Add events to be fired on block and test completions

[just-boris]

I write my own reporter and I am wondering how to detect end of testcase.
I tried to listen event test end, but according to source code afterEach hooks are invoked after this event. It affects me, because I want to know time of whole test execution, including teardown actions. Also afterEach, can emit a fail event and it a bit unexpected, because a test already report about its finish.

Moreover, test event is emitted before beforeEach hooks invocation.

What about extra event afterEach end which will be fired after afterEach hooks invocation?

[danielstjules]

I think the recommended way is to listen to the pass, fail and pending events: https://github.com/mochajs/mocha/blob/master/lib/reporters/spec.js#L49-L75 https://github.com/mochajs/mocha/blob/master/lib/reporters/list.js#L36-L53

Doing so will keep your reporter consistent with mocha's own reporters.

[just-boris]

Well I do so, but it is surprising that sometimes I can receive two pass/fail events for one testcase

[danielstjules]

Have you seen that in a lot of different situations? Off the top of my head:

$ cat test.js
it('test', function(done) {
  setTimeout(done, 10);
  setTimeout(done, 20);
});

it('test2', function(done) {
  setTimeout(done, 100);
});

$ mocha test.js


  ✓ test
  1) test
  ✓ test2 (106ms)

  2 passing (130ms)
  1 failing

  1)  test:
     Error: done() called multiple times
      at Object.<anonymous> (test.js:1:63)
      at Array.forEach (native)
      at node.js:868:3

[just-boris]

I ran that tests with xunit reporter and got following

<testsuite name="Mocha Tests" tests="2" failures="1" errors="1" skipped="-1" timestamp="Mon, 14 Sep 2015 17:58:02 GMT"
           time="0.125">
    <testcase classname="" name="test" time="0.014">
        <failure><![CDATA[done() called multiple times
Error: done() called multiple times
    at Object.&lt;anonymous&gt; (test.js:1:63)
    at Array.forEach (native)
    at node.js:814:3]]></failure>
    </testcase>
    <testcase classname="" name="test" time="0.014">
        <failure><![CDATA[done() called multiple times
Error: done() called multiple times
    at Object.&lt;anonymous&gt; (test.js:1:63)
    at Array.forEach (native)
    at node.js:814:3]]></failure>
    </testcase>
    <testcase classname="" name="test2" time="0.106"/>
</testsuite>

This report is weird. Here is a negative skipped tests count and first test mentioned twice. And both items has the same failure which makes one of them useless, because of initial test result has been overwritten.

I am trying to avoid this weirdness in my reporter, but I am not sure that is my issue rather than test runner itself.

[segrey]

+1, same issue
According to https://github.com/mochajs/mocha/blob/v2.4.5/lib/runner.js#L563, we have this behavior:

• test event is fired before beforeEach hook (IMO expectedly)
• pass, fail, pending and test end events are fired before (not after) afterEach hook (IMO unexpectedly).

For WebStorm mocha integration it caused attaching stdout printed in afterEach hook to a wrong node in a test tree. It was fixed by applying a workaround (JetBrains/mocha-intellij@03345ee). However, it'd be nice to have finalizing events (pass, fail, pending and test end) fired after afterEach hook. Hope it won't break any mocha invariant, will it?

[danlomantoSamsungNext]

I know it's been a while, but is there any plan to address the issue of having the pass, fail, and pending events be triggered after an afterEach() hook has executed?

I have some cleanup work in the afterEach() hook that i'd like to access after the test finishes execution.

[plroebuck]

I know it's been a while, but is there any plan to address the issue of having the pass, fail, and pending events be triggered after an afterEach() hook has executed?

None. Those events are test-related (i.e., whether it succeeded or not).
Trying to fail the test after the fact?

I have some cleanup work in the afterEach() hook that i'd like to access after the test finishes execution.

Can you provide an example of why your cleanup would require further event handling?
Something further than this.currentTest.state from within afterEach hook?

[danlomantoSamsungNext]

Can you provide an example of why your cleanup would require further event handling?
Something further than this.currentTest.state from within afterEach hook?

@plroebuck Sure thing!

In the afterEach() hook, I will do a couple different things:

• If a test fails, I will take a screenshot of the final state of a UI test.
• If a test fails, I will generate a link to the UI test recording (link to Browserstack).

I can't do this in each individual test because if an assertion fails, then obviously an assertion exception is thrown and I have nothing to catch that. And I'd prefer not to do that in the test as well since that would result in a lot of code duplication. And if I just listen to the pass, fail, and pending events, that data is not attached to the test object since the afterEach() hook hasn't executed yet.

Looking at other test frameworks like NUnit, their teardown function (same thing conceptually as afteEach()) is considered to be part of the test execution. It'd be nice if Mocha did the same.

That being said, I did find a work around for my scenario where I'm listening to the hook end event and filtering out only on after each events. From that event I can get the test that executed from the event context (which includes the data I'm setting on the test object in the afterEach() hook).

[lukeapage]

Another work around is to call mocha.suite.afterEach to add your own after each. if it’s done after all files are added then it will run after any user after’s.

[JoshuaKGoldberg]

Sounds like #1860 (comment) is an appropriate workaround for this feature gap? I'm not 100% confident this issue actually is resolved by that - but it's been a couple years and not that many people have interacted here. Marking as closed for now.

If someone still would like to add more events, great! Please do file a new issue that says specifically what events you'd like to add.

For reference, the event string constants in source code are here:

mocha/lib/runner.js

Line 48 in b7b849b

var constants = utils.defineConstants(

[just-boris]

Sounds like #1860 (comment) is an appropriate workaround for this feature gap?

I expected that reporter lives as a separate layer and does not mess with tests flow. But if you are saying this is the way reporters should work, ok by me ¯_(ツ)_/¯