Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2.4.5 - afterEach run multiple extra times and wrong number of tests reported if a test errors #2216

Closed
rally25rs opened this issue Apr 19, 2016 · 4 comments

Comments

@rally25rs
Copy link

rally25rs commented Apr 19, 2016

I think I've stumbled into a weird edge case here.

I have global-level beforeEach and afterEach handlers, as well as some nested in a describe, basically like this:

beforeEach(function () {
  console.log('GLOBAL BEFORE');
});

afterEach(function () {
  console.log('GLOBAL AFTER');
});

describe('', function() {
  beforeEach(function () {
    console.log('BEFORE');
  });

  afterEach(function () {
    console.log('AFTER');
  });

  it('test 1', function () {
    console.log('test 1');
  });

  it('test 2', function () {
    console.log('test 2');
  });
});

It seems that if all my tests pass, I get the output

LOG: 'GLOBAL BEFORE'
LOG: 'BEFORE'
LOG: 'test 1'
LOG: 'AFTER'
LOG: 'GLOBAL AFTER'
LOG: 'GLOBAL BEFORE'
LOG: 'BEFORE'
LOG: 'test 2'
LOG: 'AFTER'
LOG: 'GLOBAL AFTER'
LOG: 'GLOBAL BEFORE'
LOG: 'BEFORE'
LOG: 'test 3'
LOG: 'AFTER'
LOG: 'GLOBAL AFTER'
PhantomJS 1.9.8 (Mac OS X 0.0.0): Executed 91 of 91 SUCCESS (0.981 secs / 0.702 secs)

Note the test count is 91 (correct) and it measures the execution time at 0.702s.

However, if one of my tests errors, I get:

LOG: 'GLOBAL BEFORE'
LOG: 'BEFORE'
LOG: 'test 1'
LOG: 'AFTER'
LOG: 'GLOBAL AFTER'
LOG: 'GLOBAL BEFORE'
LOG: 'BEFORE'
LOG: 'test2'
LOG: 'AFTER'
LOG: 'GLOBAL AFTER'
LOG: 'GLOBAL BEFORE'
LOG: 'BEFORE'
LOG: 'test3'
PhantomJS 1.9.8 (Mac OS X 0.0.0) test 3 FAILED
    TypeError: 'undefined' is not a function (evaluating 'this.foo()') (/Users/me/Projects/foo/test/tests.js:1212)
LOG: 'AFTER'
LOG: 'AFTER'
LOG: 'GLOBAL AFTER'
LOG: 'GLOBAL AFTER'
LOG: 'GLOBAL AFTER'
PhantomJS 1.9.8 (Mac OS X 0.0.0): Executed 94 of 91 (3 FAILED) (0.969 secs / NaN secs)

Note now that after the failing "test 3" the nested afterEach is called twice, and the global afterEach is called 3 times!

Also it reports 94/91 tests now (cool, it ran 103% of my tests! :) ) and reports the execution time as NaN


I'll try to distill my actual test code into a sharable example to reproduce this behavior, but I suspect there is some weird edge case I've run into that will disappear as soon as I change anything...

@rally25rs
Copy link
Author

rally25rs commented Apr 19, 2016

I reproduced some of the issues here: https://github.com/rally25rs/mocha-issue2216

but can't seem to reproduce whatever was making the afterEach get called multiple times...

I can get some weird behavior to occur, mostly centered around async tests that throw an exception.

@rally25rs
Copy link
Author

rally25rs commented Apr 19, 2016

Not sure if this helps at all either, but with the HTML reporter / running in a browser, I see the same test triplicated (the 2 failures and last success are all the same test. there are actually only 3 tests not 5).

There is an error on the console, and it seems if you debug there, this.test.body is undefined.

screen shot 2016-04-19 at 1 00 25 pm

Related to what I point out in my example github code from the previous comment, it is actually the 2nd test in that screenshot that is throwing the error, but it causes the 3rd test to fail... which then reports itself as 2 failures and 1 success.

@rally25rs
Copy link
Author

Part of this is related to #2112

@danielstjules
Copy link
Contributor

Yup, this is a duplicate of #2083 and should be fixed in the next release, sorry!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants