Composer not pulling latest version of package from lower priority Registry/Composer Repository

[adamj88]

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us what version of Renovate you run.

34.118

If you're self-hosting Renovate, select which platform you are using.

gitlab.com

If you're self-hosting Renovate, tell us what version of the platform you run.

.com EE

Was this something which used to work for you, and then stopped?

I never saw this working

Wanted end result.

When scanning for a package version using Composer a lower version is returned from a registry with a higher priority when a newer version exists in a repository with a lower priority.

In my example, the below versions are the latest in each registry:

• 1.4.4 https://repo.magento.com/ - see: https://marketplace.magento.com/facebook-facebook-for-magento2.html#product.info.details.release_notes
• 1.4.6 https://packagist.org - see https://packagist.org/packages/facebook/facebook-for-magento2#1.4.6

The below is returned for a package during debugging the renovate package:

{
 "depType": "require",
 "depName": "facebook/facebook-for-magento2",
 "currentValue": "^1.4.4",
 "datasource": "packagist",
 "registryUrls": [
   "https://repo.magento.com/",
   "https://packagist.org"
 ],
 "depIndex": 9,
 "updates": [
   {
     "updateType": "pin",
     "isPin": true,
     "newValue": "1.4.4",
     "newVersion": "1.4.4",
     "newMajor": 1,
     "branchName": "renovate/pin-dependencies"
   }
 ],
 "warnings": [],
 "versioning": "composer",
 "registryUrl": "https://repo.magento.com",
 "currentVersion": "1.4.4"
}

I would expect this to keep scanning even though the repository is of lower priority as we have the following canonical set to false in composer, as documented here: https://getcomposer.org/doc/articles/repository-priorities.md#making-repositories-non-canonical

With the below configuration:

"repositories": [
  {
    "type": "composer",
    "url": "https://repo.magento.com/",
    "canonical": false
  }
]

I wanted to ask, is there any configuration I can change to carry on scanning the remaining registry URLs or if there is a bug in the process?

What you tried so far.

Reading the documentation and looking through the code, I've been unable to find any option which might solve this.

Relevant debug logs

Logs

Copy/paste the relevant log(s) here, between the starting and ending backticks

[RahulGautamSingh]

When there are multiple registryUrls then it depends on the datasource's registryStrategy (in this case: packagist) how they will be used.

Incase of packagist the registryStrategy is hunt

renovate/lib/modules/datasource/packagist/index.ts

Line 33 in 64fee7e

override readonly registryStrategy = 'hunt';

It means that we stop looking when the first valid result is found.

It cannot be configured via renovate-config by users currently.

Owing to the presence of canonical options I think merge will be a better fit for packagist.
Let's wait for the maintainers comment on this.👍

[adamj88]

@RahulGautamSingh thanks for your reply, it explains my issue much better :)

The merge registryStrategy certainly looks more attuned to the Composer 1 behavior and it would suit my use case, the lack of configuration in that area presents a problem however.

In Composer 2 with the canonical options, it would be more saying, for repositories (registries) marked canonical true (which is the default) then use the hunt strategy maintaining their order, but if the package is not found in those then scan the canonical false with the merge strategy.

In the meantime, are you aware of any workaround I could configure to force the package to be pulled from a specific registry? Perhaps using the packageRules configuration?

[RahulGautamSingh]

My bad for not adding a workaround in my 1st reply 😅

"packageRules": [
    {
    "matchPackageNames": ["facebook/facebook-for-magento2"],
      "registryUrls": ["https://packagist.org"]
    }
  ]

[RahulGautamSingh]

I used matchPackageNames so that other deps won't be affected, you can try other matching options as well see here

[adamj88]

@RahulGautamSingh no fault of yours at all, the workaround has allowed me to bypass the problem and upgrade that package as expected temporarily, thanks for your help 🙂

I expect to have to manually have to maintain this list should a similar set of circumstances arise, it may be difficult to know when this happens though with Renovate in place.

Would the behaviour described above be best raised as an issue instead as opposed to a discussion?

[RahulGautamSingh]

Yes, you can open an issue, mentioning your problem and the possible solution ie. using 2 strategies for composer as you describe in the previous comment. 👍

[RahulGautamSingh]

Tracked here: #20201