Inconsistent reporting results

[robtarr]

Link to bug demonstration repository.

https://github.com/robtarr/broken-nyc

Expected Behavior

Repeated runs against the same codebase should return the same results.

Observed Behavior

Different output on consecutive nyc runs

Forensic Information

MacOS Mojave 10.14.2
Node 10.11.0
https://github.com/robtarr/broken-nyc/blob/master/environment.txt

❯ npm run coverage

> [email protected] coverage .../code-coverage-test
> nyc npm test


> [email protected] test .../code-coverage-test
> mocha --require babel-core/register



  add
    ✓ adds 2 numbers


  1 passing (7ms)


=============================== Coverage summary ===============================
Statements   : 0% ( 0/3 )
Branches     : 100% ( 0/0 )
Functions    : 0% ( 0/3 )
Lines        : 0% ( 0/3 )
================================================================================

❯ npm run coverage

> [email protected] coverage .../code-coverage-test
> nyc npm test


> [email protected] test .../code-coverage-test
> mocha --require babel-core/register



  add
    ✓ adds 2 numbers


  1 passing (6ms)


=============================== Coverage summary ===============================
Statements   : 25% ( 1/4 )
Branches     : 100% ( 0/0 )
Functions    : 25% ( 1/4 )
Lines        : 25% ( 1/4 )
================================================================================

[coreyfarrell]

Please take a look at https://istanbul.js.org/docs/tutorials/es2015/. I'm able to get your repository working by following the instructions on that page. The main points:

1. Testing ES2015+ requires switching from built-in NYC instrumentation to babel-plugin-istanbul.
2. Everything is tested/supported with babel 7. I don't know if current versions will work with babel 6 but we don't support it.
3. It's important to have nyc require @babel/register. In this setup the "all": true setting is only honored if nyc actually requires babel.

It doesn't seem to create harm for mocha to also require @babel/register. Having mocha require babel is needed if you want npm test not run coverage and npm run coverage to do nyc npm test.

[coreyfarrell]

I'm closing this as it seems to be a configuration error rather than an nyc bug. If you experience issues after following the instructions I provided let us know.