test coverage is empty - v3.0.0 #6888

pete-redmond-cko · 2019-04-24T16:01:22Z

Is this a bug report?

yes, test coverage is always empty

After updating react-scripts to 3.0.0 test coverage is always empty, this worked before in v2.1.8

v2.1.8

v3.0.0

Did you try recovering your dependencies?

This happens after booting an app up using create react app using typescript create-react-app cra-coverage-example --typescript

Which terms did you search for in User Guide?

n/a

Environment

Environment Info:

  System:
    OS: macOS 10.14.4
    CPU: x64 Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz
  Binaries:
    Node: 11.12.0 - ~/.nvm/versions/node/v11.12.0/bin/node
    Yarn: 1.15.2 - /usr/local/bin/yarn
    npm: 6.7.0 - ~/.nvm/versions/node/v11.12.0/bin/npm
  Browsers:
    Chrome: 73.0.3683.103
    Safari: 12.1
  npmPackages:
    react: ^16.8.6 => 16.8.6
    react-dom: ^16.8.6 => 16.8.6
    react-scripts: 3.0.0 => 3.0.0
  npmGlobalPackages:
    create-react-app: Not Found

Steps to Reproduce

create-react-app cra-coverage-example --typescript
yarn test --coverage

Expected Behavior

test coverage would be output

Actual Behavior

test coverage is always empty

Reproducible Demo

https://github.com/pete-redmond-cko/cra-coverage-example or you can spin up an app using create react app v.3.0.0 and run yarn test --coverage

The text was updated successfully, but these errors were encountered:

bugzpodder · 2019-04-24T16:54:41Z

If you turn off watch via:
yarn test --coverage --watchAll=false

It will work properly. Looks like a Jest bug. @stipsan

esturcke · 2019-04-24T18:20:49Z

Thanks @bugzpodder !

This also fixed a regression we had running tests/coverage in Docker causing us to hit up against the fs.inotify.max_user_watches limit. Added --watchAll=false and now our tests complete again.

azeezat · 2019-04-26T08:46:17Z

If you turn off watch via:
yarn test --coverage --watchAll=false

It will work properly. Looks like a Jest bug. @stipsan

This fixed the issue

aenciso · 2019-04-26T10:15:24Z

If you turn off watch via:
yarn test --coverage --watchAll=false

It will work properly. Looks like a Jest bug. @stipsan

It doesn't fix it for me though. Still getting empty results

bugzpodder · 2019-04-26T20:24:23Z

@aenciso can you show your entire command that you ran as well as the line in scripts section of package.json that shows actual command.

aenciso · 2019-04-28T20:38:39Z

@aenciso can you show your entire command that you ran as well as the line in scripts section of package.json that shows actual command.

From command line I run
yarn test --coverage --watchAll=false

bugzpodder · 2019-04-28T21:27:26Z

I cannot repo this, please provide more details or a minimal repro demonstrating this issue.

omguhh · 2019-04-29T05:21:54Z

I'm having the same issue as @aenciso - I think it's a jest bug, it's been brought up as an issue there as well after the latest release. jestjs/jest#8364

I can run all the tests fine and see coverage if I'm using this command
npm t -- --coverage --watchAll=false

but running npm t -- --coverage --watch returns an empty table of results, so I can't really 'watch' for changes when I'm updating a test :(

mraak · 2019-04-29T11:02:53Z

Jest needs more Jest tests to test the Jest.

hugomn · 2019-05-01T11:43:42Z

I have the same issue and can confirm that --watchAll=false temporarily solves the issue. But it's not a good solution because I cannot use watch at all, together with the coverage.

bugzpodder · 2019-05-01T18:08:01Z

Can you elaborate on your last statement

hugomn · 2019-05-01T18:15:14Z

@bugzpodder the only way to have coverage displayed is disabling watch. So when you are working on tests, trying to fix coverage issues, you'll need to re-run all tests every time. That's why I said it's only a temporary solution, but the bug is still there.

bugzpodder · 2019-05-01T18:27:10Z

As a workaround, you can temporarily modify the component file (or whichever file you are trying to add coverage for) and then the coverage will show up.

aemc · 2019-05-06T19:46:23Z

I thought I was the only one... glad I found this!

bugzpodder · 2019-05-06T20:28:16Z

@aemc @hugomn did modifying the original file help with the coverage issue under --watch mode? I am trying to gauge the extend of the bug.

hugomn · 2019-05-07T11:54:23Z

@bugzpodder the coverage doesn't show at all under watch mode. Even changing files.

bugzpodder · 2019-05-07T18:53:27Z

It works fine for me, I thought it might be typescript files but that worked as well. You'll need to provide a repo.

 PASS  src/App.test.tsx (5.164s)
  ✓ renders without crashing (15ms)

-----------|----------|----------|----------|----------|-------------------|
File       |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
-----------|----------|----------|----------|----------|-------------------|
All files  |    66.67 |      100 |      100 |    66.67 |                   |
 App.tsx   |      100 |      100 |      100 |      100 |                   |
 index.tsx |        0 |      100 |      100 |        0 |                 6 |
-----------|----------|----------|----------|----------|-------------------|
Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        9.245s
Ran all test suites related to changed files.

Watch Usage
 › Press a to run all tests.
 › Press f to run only failed tests.
 › Press q to quit watch mode.
 › Press p to filter by a filename regex pattern.
 › Press t to filter by a test name regex pattern.
 › Press Enter to trigger a test run.

MrHus · 2019-05-10T11:56:21Z

Unfortunately the npm test -- --coverage --watchAll=false does not work for me as well.

bugzpodder · 2019-05-10T17:09:56Z

@MrHus please provide a repo and outputs.

tswardenski · 2019-05-10T18:35:54Z

@bugzpodder I don't have a repo to share because I'm experiencing this for a work app and it isn't open source, however this does appear to be a Jest 24 watch issue to me. Here's the same output from both 2.1.8 and 3.0.1:

☁  app [init] ⚡  node -v && npm ls jest && yarn test --coverage
v11.4.0
[email protected] /Users/tswardenski/app
└─┬ [email protected]
  └── [email protected]

yarn run v1.16.0
$ react-scripts test --coverage
 PASS  src/lib/utils/__tests__/flatten.spec.js
 PASS  src/lib/utils/__tests__/mcUserInfo.spec.js
 PASS  src/lib/utils/__tests__/sendEvent.spec.js
 PASS  src/lib/i18n/actions/__tests__/index.spec.js
 PASS  src/lib/i18n/reducers/__tests__/i18n.spec.js
 PASS  src/lib/i18n/containers/__tests__/Intl.spec.js
--------------------------|----------|----------|----------|----------|-------------------|
File                      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
--------------------------|----------|----------|----------|----------|-------------------|
All files                 |    17.45 |     5.48 |     10.6 |     17.5 |                   |
 src                      |      100 |      100 |      100 |      100 |                   |
  setupTests.js           |      100 |      100 |      100 |      100 |                   |
 src/lib/api              |        0 |        0 |        0 |        0 |                   |
  Endpoint.js             |        0 |        0 |        0 |        0 |... 60,62,72,76,77 |
  api.js                  |        0 |        0 |        0 |        0 |... 46,50,51,52,53 |
 src/lib/components       |        0 |      100 |      100 |        0 |                   |
  index.js                |        0 |      100 |      100 |        0 |               1,2 |
 src/lib/components/Error |        0 |      100 |        0 |        0 |                   |
  Error.js                |        0 |      100 |        0 |        0 |    1,2,4,5,7,8,22 |
 src/lib/i18n             |       75 |    58.82 |      100 |    76.92 |                   |
  I18nLoader.js           |    77.78 |    58.82 |      100 |       80 |    18,20,41,45,62 |
  index.js                |        0 |      100 |      100 |        0 |                 1 |
 src/lib/i18n/actions     |      100 |        0 |      100 |      100 |                   |
  index.js                |      100 |        0 |      100 |      100 |                 4 |
 src/lib/i18n/constants   |      100 |      100 |      100 |      100 |                   |
  index.js                |      100 |      100 |      100 |      100 |                   |
 src/lib/i18n/containers  |    14.29 |        0 |        0 |    14.29 |                   |
  Intl.js                 |    14.29 |        0 |        0 |    14.29 |... 31,40,41,68,76 |
 src/lib/i18n/reducers    |      100 |      100 |      100 |      100 |                   |
  i18n.js                 |      100 |      100 |      100 |      100 |                   |
 src/lib/static           |    13.08 |        0 |        0 |    13.18 |                   |
  mc-lib.js               |    13.08 |        0 |        0 |    13.18 |... 1530,1535,1540 |
 src/lib/utils            |       90 |    88.24 |      100 |       90 |                   |
  flatten.js              |      100 |      100 |      100 |      100 |                   |
  index.js                |      100 |      100 |      100 |      100 |                   |
  mcUserInfo.js           |    77.78 |    71.43 |      100 |    77.78 |              9,13 |
  sendEvent.js            |      100 |      100 |      100 |      100 |                   |
--------------------------|----------|----------|----------|----------|-------------------|

Test Suites: 6 passed, 6 total
Tests:       13 passed, 13 total
Snapshots:   0 total
Time:        4.313s
Ran all test suites.
✨  Done in 6.34s.

☁  app [init] ⚡  node -v && npm ls jest
v11.4.0
[email protected] /Users/tswardenski/app
└─┬ [email protected]
  └── [email protected]
☁  app [init] ⚡  yarn test --coverage
No tests found related to files changed since last commit.
Press `a` to run all tests, or run Jest with `--watchAll`.
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|

Watch Usage
 › Press a to run all tests.
 › Press f to run only failed tests.
 › Press q to quit watch mode.
 › Press p to filter by a filename regex pattern.
 › Press t to filter by a test name regex pattern.
 › Press Enter to trigger a test run.

and when pressing a to watch all:

 PASS  src/lib/utils/__tests__/sendEvent.spec.js
 PASS  src/lib/utils/__tests__/flatten.spec.js
 PASS  src/lib/utils/__tests__/mcUserInfo.spec.js
 PASS  src/lib/i18n/actions/__tests__/index.spec.js
 PASS  src/lib/i18n/reducers/__tests__/i18n.spec.js
 PASS  src/lib/i18n/containers/__tests__/Intl.spec.js
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|

Test Suites: 6 passed, 6 total
Tests:       13 passed, 13 total
Snapshots:   0 total
Time:        3.167s
Ran all test suites.

Watch Usage: Press w to show more.

☁  app [init] ⚡  node -v && npm ls jest && yarn test --coverage --watchAll=false
v11.4.0
[email protected] /Users/tswardenski/app
└─┬ [email protected]
  └── [email protected]

yarn run v1.16.0
$ react-scripts test --coverage --watchAll=false
 PASS  src/lib/utils/__tests__/flatten.spec.js
 PASS  src/lib/i18n/actions/__tests__/index.spec.js
 PASS  src/lib/utils/__tests__/sendEvent.spec.js
 PASS  src/lib/utils/__tests__/mcUserInfo.spec.js
 PASS  src/lib/i18n/reducers/__tests__/i18n.spec.js
 PASS  src/lib/i18n/containers/__tests__/Intl.spec.js
------------------|----------|----------|----------|----------|-------------------|
File              |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
------------------|----------|----------|----------|----------|-------------------|
All files         |    17.43 |     5.48 |     10.6 |    17.59 |                   |
 api              |        0 |        0 |        0 |        0 |                   |
  Endpoint.js     |        0 |        0 |        0 |        0 |... 60,62,72,76,77 |
  api.js          |        0 |        0 |        0 |        0 |... 46,50,51,52,53 |
 components       |        0 |        0 |        0 |        0 |                   |
  index.js        |        0 |        0 |        0 |        0 |                   |
 components/Error |        0 |      100 |        0 |        0 |                   |
  Error.js        |        0 |      100 |        0 |        0 |            7,8,22 |
 i18n             |    77.78 |    58.82 |      100 |       80 |                   |
  I18nLoader.js   |    77.78 |    58.82 |      100 |       80 |    18,20,41,45,62 |
  index.js        |        0 |        0 |        0 |        0 |                   |
 i18n/actions     |      100 |        0 |      100 |      100 |                   |
  index.js        |      100 |        0 |      100 |      100 |                 4 |
 i18n/constants   |      100 |      100 |      100 |      100 |                   |
  index.js        |      100 |      100 |      100 |      100 |                   |
 i18n/containers  |    14.29 |        0 |        0 |    14.29 |                   |
  Intl.js         |    14.29 |        0 |        0 |    14.29 |... 31,40,41,68,76 |
 i18n/reducers    |      100 |      100 |      100 |      100 |                   |
  i18n.js         |      100 |      100 |      100 |      100 |                   |
 static           |       13 |        0 |        0 |    13.18 |                   |
  mc-lib.js       |       13 |        0 |        0 |    13.18 |... 1530,1535,1540 |
 utils            |       90 |    88.24 |      100 |       90 |                   |
  flatten.js      |      100 |      100 |      100 |      100 |                   |
  index.js        |        0 |        0 |        0 |        0 |                   |
  mcUserInfo.js   |    77.78 |    71.43 |      100 |    77.78 |              9,13 |
  sendEvent.js    |      100 |      100 |      100 |      100 |                   |
------------------|----------|----------|----------|----------|-------------------|

Test Suites: 6 passed, 6 total
Tests:       13 passed, 13 total
Snapshots:   0 total
Time:        3.953s, estimated 4s
Ran all test suites.
✨  Done in 5.60s.

// package.json
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject",
    "flow": "flow"
  }

bugzpodder · 2019-05-10T18:52:34Z

@tswardenski thanks for the repo. There seems to be two things here:

when you run yarn test --coverage, it doesn't show coverage details because no test files was detected and you got the message No tests found related to files changed since last commit.
This is generally expected behavior, because no test is run you don't get coverage info.
After step 1., you press 'a' to run all tests, however still there are no coverage info showing up. This looks like a bug to me. The workaround is to run it with --watchAll=false

tswardenski · 2019-05-10T19:40:08Z

@bugzpodder I have made changes to both specfiles and JS files between commits and run the command and while it does run the test(s) for the uncommitted changes, the coverage report still displays an empty table.

bugzpodder · 2019-05-10T20:00:30Z

@tswardenski can you show me the output of yarn test --coverage in this case? Thanks.

sketchbuch · 2019-05-11T09:56:10Z

I have the same problem (v3.0.1). I can't run coverage at all. It used to work, this is my repo: https://github.com/sketchbuch/school-report

when I run > npm run test:c

it says no tests have changed. --watchAll or --watchAll=false does nothing. Pressing "a" runs the tests but produces no coverage or even runs coverage. The coverage folder is created but contains just a few files - mostly little images used in the report display.

bugzpodder · 2019-05-11T14:54:35Z

@sketchbuch For npm, you'll need an extra set of dashes to get watchAll flag to work, something like: npm run test:c -- --watchAll=false, which produces result:

-----------------------------------------|----------|----------|----------|----------|-------------------|
File                                     |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
-----------------------------------------|----------|----------|----------|----------|-------------------|
All files                                |    56.75 |    47.32 |    47.87 |    57.32 |                   |
 src                                     |        0 |        0 |        0 |        0 |                   |
  index.js                               |        0 |      100 |      100 |        0 |                10 |
  registerServiceWorker.js               |        0 |        0 |        0 |        0 |... 87,91,96,97,98 |
 src/js/actions                          |    98.16 |    81.82 |    98.68 |    98.16 |                   |
  appActions.js                          |      100 |      100 |      100 |      100 |                   |
  builderActions.js                      |      100 |      100 |      100 |      100 |                   |
  categoryActions.js                     |      100 |      100 |      100 |      100 |                   |
  classActions.js                        |      100 |      100 |      100 |      100 |                   |
  dataActions.js                         |      100 |      100 |      100 |      100 |                   |

Here I've changed one file, and the coverage also works correctly:

git diff
diff --git a/src/js/actions/appActions.js b/src/js/actions/appActions.js
index 225a476..a5513a5 100644
--- a/src/js/actions/appActions.js
+++ b/src/js/actions/appActions.js
@@ -1,4 +1,5 @@
 // @flow
+console.log('foo');
 
 import { APP_ERRORED, APP_LOADED } from '../constants/actionTypes';
 import type { ActionCreator } from '../types/action';

 PASS  src/js/actions/tests/appActions.test.js
  ● Console

    console.log src/js/actions/appActions.js:2
      foo

 PASS  src/js/components/App/App.test.js
  ● Console

    console.log src/js/actions/appActions.js:2
      foo

---------------|----------|----------|----------|----------|-------------------|
File           |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
---------------|----------|----------|----------|----------|-------------------|
All files      |      100 |      100 |      100 |      100 |                   |
 appActions.js |      100 |      100 |      100 |      100 |                   |
---------------|----------|----------|----------|----------|-------------------|

Test Suites: 2 passed, 2 total
Tests:       5 passed, 5 total
Snapshots:   0 total
Time:        3.256s, estimated 6s
Ran all test suites related to changed files.

Watch Usage
 › Press a to run all tests.
 › Press f to run only failed tests.
 › Press q to quit watch mode.
 › Press p to filter by a filename regex pattern.
 › Press t to filter by a test name regex pattern.
 › Press Enter to trigger a test run.

sketchbuch · 2019-05-11T15:53:24Z

@bugzpodder: Thanks! that got coverage working again 👍

tswardenski · 2019-05-13T13:42:48Z

@bugzpodder Here you go. v3 with Jest 24.7 running yarn test --coverage

 PASS  src/lib/i18n/containers/__tests__/Intl.spec.js
  <Intl />
    ✓ should render the container without crashing (41ms)
    ✓ should render the container without crashing (5ms)

----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|
Test Suites: 1 passed, 1 total
Tests:       2 passed, 2 total
Snapshots:   0 total
Time:        3.615s
Ran all test suites related to changed files.

Watch Usage
 › Press a to run all tests.
 › Press f to run only failed tests.
 › Press q to quit watch mode.
 › Press p to filter by a filename regex pattern.
 › Press t to filter by a test name regex pattern.
 › Press Enter to trigger a test run.

bugzpodder · 2019-05-13T16:50:48Z

@tswardenski in your example, I just want to double check that you modified the original source code src/lib/i18n/containers/Intl.js instead of src/lib/i18n/containers/tests/Intl.spec.js?
It's a known issue that if you just modify the test file Intl.spec.js, it isn't going to show the coverage of original source code.

tswardenski · 2019-05-13T17:00:57Z

Ah, yes. Editing the original source code does now include the changed file in the coverage output.

amaankulshreshtha · 2019-05-14T15:46:14Z

Is there a way to --watch and --watchAll=false at the same time, because the above combination doesn't work

bugzpodder · 2019-05-14T15:49:36Z

What doesn't work? Watch and watchAll=false does the exact opposite.

amaankulshreshtha · 2019-05-14T16:49:58Z

Is there a way, to put it in watch mode without having the error?

bugzpodder · 2019-05-14T16:59:07Z

I have no idea which error you are referring to. Please provide a repro on the error you are running into.

amaankulshreshtha · 2019-05-14T17:06:04Z

I was facing the same issue, where the coverage was not showing anything, despite setting the collectCoverageFrom property to the absolute path of the js file.
As per your answer above: #6888 (comment), this got resolved, but I have to run the test command every time I make any changes. So is there a solution where I don't have to manually trigger the tests.

bugzpodder · 2019-05-14T17:17:26Z

As you yourself suggested, if you don't want to manually trigger tests you need to run jest in watch mode. Hence yarn test --coverage or npm test -- --coverage should show the coverage of a source file provided that you made some modifications to it and it gets picked up by the watch mode.

madeleinemf · 2019-05-22T15:15:44Z

I am still getting no coverage when running npm test -- --coverage (react-scripts v3.0.1). The workaround of running with --watchAll=false does work, but is a little annoying.

davidnguyen11 · 2019-05-23T08:09:07Z

try this one

npm run test -- --coverage --watchAll=false

bugzpodder · 2019-05-23T16:17:49Z

At this point, I haven't seen a legit bug except for in a minor case (pressing 'a' in watch mode doesn't retroactive show coverage). In all cases the feature is working as expected. If you disagree, please file a new issue with a clean repro so we can investigate. Thanks!

mwibutsa · 2019-05-28T09:19:54Z

Thank you --watchAll worked for me

bugzpodder added the issue: needs investigation label Apr 24, 2019

hodgef added a commit to DevUnltd/js-library-boilerplate that referenced this issue Apr 28, 2019

Added working coverage command. Per facebook/create-react-app#6888

953288a

lugithub referenced this issue in lugithub/rabbit May 21, 2019

kent example

0c16077

bugzpodder closed this as completed May 23, 2019

bugzpodder mentioned this issue May 23, 2019

Run jest in --watch --coverage doesn't update coverage info when files change jestjs/jest#8491

Closed

lock bot locked and limited conversation to collaborators Jun 2, 2019

test coverage is empty - v3.0.0 #6888

test coverage is empty - v3.0.0 #6888

Comments

pete-redmond-cko commented Apr 24, 2019

Is this a bug report?

Did you try recovering your dependencies?

Which terms did you search for in User Guide?

Environment

Steps to Reproduce

Expected Behavior

Actual Behavior

Reproducible Demo

bugzpodder commented Apr 24, 2019

esturcke commented Apr 24, 2019

azeezat commented Apr 26, 2019

aenciso commented Apr 26, 2019

bugzpodder commented Apr 26, 2019

aenciso commented Apr 28, 2019

bugzpodder commented Apr 28, 2019

omguhh commented Apr 29, 2019 • edited Loading

mraak commented Apr 29, 2019

hugomn commented May 1, 2019

bugzpodder commented May 1, 2019

hugomn commented May 1, 2019

bugzpodder commented May 1, 2019

aemc commented May 6, 2019

bugzpodder commented May 6, 2019

hugomn commented May 7, 2019

bugzpodder commented May 7, 2019

MrHus commented May 10, 2019

bugzpodder commented May 10, 2019

tswardenski commented May 10, 2019 • edited Loading

bugzpodder commented May 10, 2019 • edited Loading

tswardenski commented May 10, 2019

bugzpodder commented May 10, 2019

sketchbuch commented May 11, 2019 • edited Loading

bugzpodder commented May 11, 2019

sketchbuch commented May 11, 2019

tswardenski commented May 13, 2019 • edited Loading

bugzpodder commented May 13, 2019

tswardenski commented May 13, 2019

amaankulshreshtha commented May 14, 2019

bugzpodder commented May 14, 2019

amaankulshreshtha commented May 14, 2019

bugzpodder commented May 14, 2019

amaankulshreshtha commented May 14, 2019 • edited Loading

bugzpodder commented May 14, 2019

madeleinemf commented May 22, 2019

davidnguyen11 commented May 23, 2019

bugzpodder commented May 23, 2019

mwibutsa commented May 28, 2019

omguhh commented Apr 29, 2019 •

edited

Loading

tswardenski commented May 10, 2019 •

edited

Loading

bugzpodder commented May 10, 2019 •

edited

Loading

sketchbuch commented May 11, 2019 •

edited

Loading

tswardenski commented May 13, 2019 •

edited

Loading

amaankulshreshtha commented May 14, 2019 •

edited

Loading