Support for npm 8.3 Overrides

[karlhorky]

What would you like Renovate to be able to do?

Hi there, first of all, thanks for your continued effort on Renovate! It's such an amazing tool, so valuable.

Similar to #1318 (updating Yarn Resolutions in package.json), it would be great if Renovate also supported npm Overrides (introduced in npm 8.3) in the same way - eg. when the version in package.json under the "overrides" object matches the version of the dependency in dependencies or devDependencies.

If you have any ideas on how this should be implemented, please tell us here.

I'm assuming that the MVP of this could be the same feature as implemented in 163ce43 (based on the latest logic in lib/modules/manager/npm/update/dependency/index.ts), reusing much of the code

However, I also see that there is a depType which may be set to resolutions, so this may also be a consideration for this change (should there be a new depType called overrides?):

renovate/lib/modules/manager/npm/update/dependency/index.ts

Lines 60 to 64 in 868ebbe

	if (match && depType === 'resolutions') {
	const patch = oldValue.replace(match[0], `${match[1]}${newValue}#`);
	parsedContents[depType]![depName] = patch;
	newString = `"${patch}"`;
	}

Is this a feature you are interested in implementing yourself?

No

---

Workaround

For now, I'm just duplicating the "overrides" data to "resolutions" and relying on the Yarn Resolutions feature of Renovate bot. In order to automatically update "overrides", I wrote this GitHub Actions workflow: https://github.com/upleveled/changes-codealong/blob/main/.github/workflows/copy-resolutions-to-overrides.yml

See example PR here: upleveled/changes-codealong#62

[github-actions]

Hi there,

Help us by making a minimal reproduction repository.

Before we can start work on your issue we first need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction to understand what is needed.

We may close the issue if you (or someone else) have not provided a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

[rarkins]

Can you provide a minimal reproduction which demonstrates a "wrong" PR but which should be updating overrides?

[karlhorky]

upleveled/changes-codealong#62 is a minimal repro PR (also above in the Workaround section) - the first commit shows the "wrong" PR commit (only updating the "resolutions" key) and the second commit is my GitHub Actions workflow that copied the object to the "overrides" key in package.json:

[rarkins]

Please check the above link for what we mean by minimal. The example PR you gave updates 10+ packages, which makes it complex to debug during development

[karlhorky]

Ok, here's a reproduction repo PR with a single dep: https://github.com/karlhorky/renovate-repro-npm-overrides/pull/3/files

[PhilipAbed]

"overrides" is supposed to be used for Transitive Dependencies,
but in the Minimal Reproduction Repository u have it in both devDeps and Resolutions too.

"devDependencies": {
   "@types/react": "18.0.6"
 },
 "resolutions": {
   "@types/react": "18.0.6"
 },

this does not look like a real usage.

Anyway, Overrides should be treated the same way that we treat dependencies/devDependencies, as it's an actual transitive dependency that is effectively used.

it's pretty easy to get a reproduction data, anything with overrides will do.

[karlhorky]

"overrides" is supposed to be used for Transitive Dependencies, but in the Minimal Reproduction Repository u have it in both devDeps and Resolutions too.

this does not look like a real usage.

It's possible that users have a direct dependency on something as well as it being in transitive dependencies. Sometimes it's really important to have the same version (such as with ESLint shared configs + plugins).

So this can indeed be a real-world use case.

[PhilipAbed]

if you want something as a direct dependency then you put it in dependencies/devDependencies

If the dependency is in Transitive as well then you add a path to it like this:

i guess its also used like you said you are right

[rarkins]

I have a feeling that was a convention we used for workspaces to designate that it's a resolution the user wants updated

[karlhorky]

used for workspaces

Right, this is another usecase - we are using Yarn Resolutions in a monorepo (with Yarn Workspaces) which may have @types/react as a transitive dep in some part of the tree (but we also need @types/react at the root). Without overriding all versions of this package to be the same version, it can cause some very confusing TS errors.

[hasanwhitesource]

@rarkins Should we create group names for children of packages ?
for example :

{
  "overrides": {
    "bar": {
      "foo": "1.0.0"
    }
  }
}

Should we in this case make foo have a groupName of bar ?

[hppycoder]

How did you get NPM 8 to run which is part of Node 16? I was under the impression that renovate is at 14.x using Kubernetes install path.

[renovate-release]

🎉 This issue has been resolved in version 32.60.0 🎉

The release is available on:

• GitHub release
• 32.60.0

Your semantic-release bot 📦🚀

[karlhorky]

Just tested 32.60.0 and it's working!

It updated the "overrides" key in my package.json in this PR:

https://github.com/upleveled/changes-codealong/pull/85/files

[rarkins]

@karlhorky awesome, thanks for confirming!

[karlhorky]

Oh, I tricked myself - it's actually not working - it was my GitHub Actions workflow script 😩

You can see the problem here - first of all Renovate bot updates the dependencies (without "overrides"), which fails the tests and also creates an "Artifact update problem":

Then, further down, my GitHub Actions workflow script (which I thought I had disabled) comes along and fixes it, which caused the PR to be ok and have the changes to "overrides".

I've now triggered the Renovate bot to open a new PR, which doesn't change the "overrides": upleveled/changes-codealong#85

[karlhorky]

@hasanwhitesource @rarkins is this PR above enough of a reproduction to figure out what's going on?

Happy to retrigger this PR anytime there are bot updates published to check if they work...

[rarkins]

Reopened the issue, @hasanwhitesource can you take a look? Assuming that you tested it on a demo repo, it could be good to share and compare that against @karlhorky's repo

[karlhorky]

One interesting side note (not sure if this is relevant) is that there are a bunch of other dependencies showing up in my dependency dashboard now (not being combined with the non-major pull requests that the bots are creating):

Wonder if this is related at all...

[karlhorky]

Oh wait, maybe there was a new dependency type called 'overrides' added?

Ahh, I need to update my shared config to also include this new 'overrides' dependency type....

Edit: Done karlhorky/renovate-config@30d6a6d

[karlhorky]

Yes, this was the problem 🤦‍♂️ Sorry for the noise!

It's indeed working, closing this again 👍