Running tests that match string causes non-matching suites to fail

[Crispioso]

💥 Regression Report

When I set the -t CLI option to filter what tests are run, if there is a suite that doesn't have any tests that match the string then it fails with:

 ● Test suite failed to run

    Your test suite must contain at least one test.

      at node_modules/jest/node_modules/jest-cli/build/test_scheduler.js:245:22

My expectation would be that it behaves the same as v22, where any suites that have no matching tests are skipped.

Sorry if this issue already exists, I looked for it but couldn't find anything that matched this problem. I've also looked in the v23 changelog and couldn't see this being a known breaking change.

Last working version

Worked up to version: 22.4.4

Stopped working in version: 23.0.0

To Reproduce

Steps to reproduce the behavior:
Clone my repo (https://github.com/Crispioso/possible-jest-bug) and run the following commands in both /v22 and /v23 directories:

npm install
npm test

They both run the same tests and apply -t false so that it only runs tests with the string 'false' in their title.

v22 skips the suite which has no matching tests whereas v23 throws an error instead.

Expected behavior

v23 to skip test suites that have no tests in them that match the -t patter, rather than error.

Link to repl or repo (highly encouraged)

https://github.com/Crispioso/possible-jest-bug

Run npx envinfo --preset jest

Paste the results here:

v23:

  System:
    OS: macOS Sierra 10.12.6
    CPU: x64 Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz
  Binaries:
    Node: 8.9.3 - /usr/local/bin/node
    Yarn: 1.0.2 - /usr/local/bin/yarn
    npm: 5.5.1 - /usr/local/bin/npm
  npmPackages:
    jest: 23.0.0 => 23.0.0

v22:

  System:
    OS: macOS Sierra 10.12.6
    CPU: x64 Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz
  Binaries:
    Node: 8.9.3 - /usr/local/bin/node
    Yarn: 1.0.2 - /usr/local/bin/yarn
    npm: 5.5.1 - /usr/local/bin/npm
  npmPackages:
    jest: 22.x => 22.4.4

[SimenB]

@rickhanlonii ideas?

[Crispioso]

By the way, I'd love to try and open a PR to fix this, but I'd probably need a bit of guidance on where to start in the codebase.

[rickhanlonii]

Oh that's a great find @Crispioso, I confirmed the bug with your repo

To fix:

• Clone the repo
• Run yarn
• Run cd packages/jest-cli && yarn link
• Go to your v23 dir
• Run yarn link jest-cli
• You're now running your local jest clone on the bug repo, run the v23 tests to be sure
• Go back to the jest repo and start yarn watch, this will automatically rebuild the source on change
• Start dropping console logs or debuggers at this line
• You'll see that testResult.testResults is empty, but it should have length 1 where the only item in the array has a status of skipped. Somewhere between those two releases we started filtering the test out of the results completely instead of including it as skipped in the results

[Crispioso]

Thanks @rickhanlonii, I'll try take a look at this over the weekend then.

[rickhanlonii]

Let me know if I can help!

[Crispioso]

@rickhanlonii I'm having problems making edits to test_scheduler.js, whenever I save the file it reformats itself (is it the prettier library doing this?) but then won't compile. I notice that the version of prettier was updated recently, could this be the cause or am I doing something wrong?

The error:

SyntaxError: ...github.com/facebook/jest/packages/jest-cli/src/test_scheduler.js: Unexpected token (52:37)
  50 | TestRunner;
  51 |
> 52 | export type TestSchedulerOptions = { |
     |                                      ^
  53 |   startRun: (globalConfig: GlobalConfig) => * ,
  54 |   |
  55 | };

Happy to post this question elsewhere as I'm aware it's not directly related to the original issue.

[SimenB]

You need to run watch mode to compile - the source code contains flow which is not valid js

[Crispioso]

I ran yarn watch, which builds fine initially until I make an edit to the file and then it throws the above error.

[Crispioso]

Okay so I think it's VSCode that is causing me problems, I'm not sure why or how but the same issues don't happen when I edit it in a simple text editor 🤔

[kentcdodds]

I'm not sure whether it's VSCode causing your issues, but for me it doesn't make a difference. This reproduces for me 100% of the time.

[vanschroeder]

I'm getting this issue with npm test -- -u -t="test string" for all published versions of v23.x. My project is not using yarn and would prefer not to add another dependency so can not use the solution posted above

It's worth noting that this only seems to be an issue with jest version >= 23.1 Rolling back to jest v22.4.4 fixes this. However, rolling back will cause flags to go up with david-dm so is less than ideal, so using jest-cli 24.x for test filtering with regex for now while leaving the package version at 23.2 as a work around

[kentcdodds]

lol, after reading things more carefully I now realize that the VSCode reference was entirely unrelated to the bug itself. Sorry about that! 😅

You can disable auto-formatting in your settings: "editor.formatOnSave": false

[thymikee]

Closed via #6657

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.