Support maxWorkers or runInBand when using 'projects'

[nonara]

🐛 Bug Report

When defining multiple projects via the projects config option, where one project has maxWorkers set (ie. 1), that setting is not respected.

In the case of a mono-repo, assume we have two projects. One which runs parallel, and another which needs to be run sequentially. For this, the latter has the config setting maxWorkers: 1.

When we run jest from the base config, however, the latter runs in parallel, ignoring the maxWorkers: 1 setting.

To Reproduce

1. Setup a mono-repo with two jest config files
2. Set one to maxWorkers: 1
3. Add both config paths to base projects property
4. Run jest

Expected behavior / Proposal

I propose the following:

1. Determine global max workers as:
   • (If present) maxWorkers setting in base config
   • (Otherwise) Determine the maximum value specified for maxWorkers found in all projects config files
2. Allow each project to use
   • (If present) Its own maxWorkers # of workers
   • (Otherwise) The global maximum # of workers

Effectively, this ensures that maxWorkers behaviour matches that of running jest on an individual project.

envinfo

  System:
    OS: Windows 10 10.0.19041
    CPU: (4) x64 Intel(R) Core(TM) i7-4510U CPU @ 2.00GHz
  Binaries:
    Node: 14.13.1 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.4 - ~\AppData\Roaming\npm\yarn.CMD
    npm: 6.14.4 - C:\Program Files\nodejs\npm.CMD
  npmPackages:
    jest: ^26.5.3 => 26.6.2

[thymikee]

Can you put up a minimal repro that could be downloaded?

[thymikee]

maxWorkers is currently a part of GlobalConfig so this is expected. I'm not sure if Jest should allow projects to limit the number of workers, as this may quickly go out of hand, causing inconsistencies between execution environments. The only scenario I see this could be helpful is limiting to 1 worker, in other words allowing runInBand to be configurable in a project. E.g. for E2E tests running on a device/simulator where parallelization doesn't make sense (unless having multiple devices).

@SimenB @jeysal thoughts?

[nonara]

The only scenario I see this could be helpful is limiting to 1 worker, in other words allowing runInBand to be configurable in a project.

That's a good point. I can think of one other use case, but I think there may be a better proposal to cover that (detail to follow).

I can see it being useful to allocate a smaller fraction of the overall worker threads in some instances.

Say we have that scenario in a very large mono-repo with multiple projects. Several config files contain very long-running tests, and those particular tests hit first. In this scenario, we might want to limit the total threads for the longer running config file so that we can run our faster tests in parallel along-side of them. This way it can fail more quickly if there are problems in the shorter running tests and avoid having to run separate calls to jest to accomplish this.

It's maybe not a perfect example, but it should give an idea.

In that scenario, we could specify a max of 3 threads to the long running out of the global 8.

However...

Amended Proposal

I think you're right about runInBand. All things considered, I'd amend my proposal to the following:

1. maxWorkers remains a global setting, allowing you to define max threads for the entire process
2. Introduce: runInBand on the project-level (This would make several folks happy Allow specifying --runInBand in config file #6709 Feature Request: Add --runInBand to package.json options #3215)
3. Introduce: minWorkers on the project-level

minWorkers

Allows configuring the minimum number of threads to be used for a particular project. (This setting would only have effect when run in projects mode)

This would allow giving specific projects the ability to use more of the allotted threads while allowing elasticity when threads are free.

So in our example above, we could configure minWorkers: 5 for our faster tests and set a global maxWorkers: 8

This would effectively give us a 3 / 5 distribution unless the faster tests finish first, in which case the longer tests will fill up the remaining slots.

[nonara]

Can you put up a minimal repro that could be downloaded?

@thymikee Did you still need this?

[thymikee]

@thymikee Did you still need this?

Nah, we're good

[flybayer]

This would be amazing!!

Our use case is fullstack testing for Blitz apps. We want client code tests to run in parallel but run server tests in series because they use the same test DB.

[taylorhoward92]

This would be amazing!!

Our use case is fullstack testing for Blitz apps. We want client code tests to run in parallel but run server tests in series because they use the same test DB.

Very similar use case here. Want to run all tests with unit tests running in parralel while integration tests run in series.

[Narretz]

Considering there's a super simple runner that provides this functionality: https://github.com/gabrieli/jest-serial-runner it shouldn't be too difficult to add this option to the project config.

[kael-shipman]

I share @taylorhoward92's use-case exactly. I thought I had come up with a solution (here), and indeed it does seem to work for me, but apparently I'm just getting lucky :).

[freshollie]

I've got a draft PR for this, thought it'll need updating: #10912

[theseyi]

maxWorkers is currently a part of GlobalConfig so this is expected. I'm not sure if Jest should allow projects to limit the number of workers, as this may quickly go out of hand, causing inconsistencies between execution environments. The only scenario I see this could be helpful is limiting to 1 worker, in other words allowing runInBand to be configurable in a project. E.g. for E2E tests running on a device/simulator where parallelization doesn't make sense (unless having multiple devices).

@SimenB @jeysal thoughts?

runInBand at the project level is definitely a very welcome compromise. I've just moved a monorepo project to use Jest's projects instead of manually invoking with yarn workspaces foreach -p run test --runInBand because I want to collect consolidate coverage for the entire workspace. Unfortunately using yarn workspaces foreach is ~1.8x - 2x faster than invoking yarn jest which is significant in economic cost and cumulative time.

[freshollie]

Finally updated my PR to implement this. Would be great to get a review :)

[freshollie]

@thymikee Is it possible someone could take a look at the PR for this? It may have been buried as it was first opened a year ago

[aidantorrence]

how about all cli options? seems ridiculous that this is not all configurable. why is maxWorkers in the config documentation but doesn't work?

[github-actions]

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

[rdsedmundo]

Not stale. There's an open PR for this.

[github-actions]

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

[rdsedmundo]

Not stale.

[freshollie]

FYI I've closed my PR related to this because I'm no longer interested in maintaining the PR state. I try not to use Jest anymore, partly due to the lack of response to my contribution/interesting in responding to feature requests like this.

If someone else wants to rebase the work on the PR and re-raise feel free.