[Feature] flag for workspaces foreach to filter to packages changed since a commit (--since?)

[seansfkelley]

• I'd be willing to implement this feature (contributing guide)
• This feature is important to have in this repository; a contrib plugin wouldn't do

Describe the user story

I work on a large Typescript monorepo. When pulling changes from my teammates, I want to rebuild as little as possible. I currently string together a custom Yarn plugin with workspaces foreach to do this:

$ yarn changes-since --as-yarn-flag <commit-ish>
--include {@foo/bar,@foo/baz,@quux/corge}
$ yarn workspaces foreach --topographical-dev $(yarn changes-since <commit-ish>) run build

This gets the job done, but is rather non-obvious, unergonomic and an odd usage of a plugin.

Describe the solution you'd like

It would be faster, simpler, more ergonomic and self-documenting to be able to do this in a single command, perhaps like:

$ yarn workspaces foreach --topographical-dev --since <commit-ish> run build

The choice of --since was borrowed from Lerna and it seems nicely evocative.

In addition to combining nicely with --topographical(-dev), it would also work well with --recursive.

As currently implemented, foreach without --recursive (with or without --from) runs the command on all selected workspaces and their nested workspaces. One can use --include/--exclude to further refine the selected set of workspaces.

To be useful, --since would have to behave more like --include than --from, but distinct from either. If --since were to include nested workspaces (like an unadorned foreach does, or --from does), then trivial changes to a nested workspace mean that --since will end up selecting not just the changed workspace, but also its parent and all its siblings, and its siblings' descendants. This defeats the purpose of the more-targeted behavior of --since.

Therefore, I think of --since as generating a preliminary whitelist of workspaces (not workspaces roots, like --from), which can be added to with --recursive and further refined with --include and --exclude.

Describe the drawbacks of your solution

As proposed, this would only work with Git. However, there is precedent in Yarn for working specifically with Git, such as the init command.

The interaction of --since with --from/--all is not clear, and I would be tempted to consider providing both an illegal invocation. Currently, the package selection process works like this:

1. --from, --all or unspecified: select the initial set of root packages
2. --recursive or unspecified: add packages according to dependencies (former) or nesting (latter)
3. --include/--exclude: if provided, whitelist/blacklist selected packages

But, since --since would have to be more like --include than --from to be useful, the package selection process gets a little more involved and branching:

1. --since is provided: select initial packages
   1. --recursive or unspecified: add packages according to dependencies (former) or do nothing (latter)
2. --since is not provided
   1. --from, --all or unspecified: as above
   2. --recursive or unspecified: as above
3. --include/--exclude: as above

I think the weirdness here stems partly from --recursive, which in the current implementation of foreach acts like an exclusionary filter, not an inclusionary one. This behavior is reversed in the presence of --since.

(I wonder if there is a related proposal hidden here to make --recursive mutually exclusive with some new --nested flag (for the current behavior) and --all. That would allow --recursive to always be additive and un-branch the selection process when --since is involved.)

Describe alternatives you've considered

As noted above, this can be strung together with a custom plugin.

My initial thought was to have --since be more like --from than --include in that it would select workspace roots, but as described earlier, this tends to make it more or less useless for targeted selection of packages.

[merceyz]

Duplicate of #2374