Remove hostnames from the lockfiles #5892
Comments
|
@arcanis this idea is being tracked in the following RFC - yarnpkg/rfcs#64 |
|
What about custom registries/repositories? If a project has a mix of different registries (eg. some from regular public npm, some from a private internal server), how would it know which is which? Particularly since there could be overlap in package names. |
|
@Daniel15 I think in this case it is recommended to keep the packages coming from a private internal server behind a specific scope, so that you can use scope-specific registry urls. |
|
Adding a datapoint here for @arcanis. At least for VSTS/TFS, this pattern seems to work. My yarn.lock matches the pattern for upstream packages: Be sure to keep scoped packages in mind - there is an extra segment: |
|
Looks like it'd work for Nexus Repository too - you have a repo section after the server root but the tail of the URL is still the same: |
|
Confirming that it looks like it would work for artifactory too. |
|
Verdaccio follows the same tarball url pattern as main registry using the default configuration, unless a user decides to use a prefix for reverse proxy. Examples from |
|
Unfortunately we discovered that some registries follow entirely different rules (a surprising one being ... NPM Enterprise ...), so we won't be able to just trim the hostname 😞 Maybe we'll end up having a whitelist of domain names that can work this way ... but it's annoying. |
|
What are their rules? |
|
Currently, it's something similar to this (cf pnpm/pnpm#867): No idea why they chose to break their convention. |
|
What about a compromise to this? Allowing for the e.g., yarn --registry=http://registry.npm.taobao.orgThis would allow the user to specify the registry to be used during the installation. Another might be to trim the hostname if it matches the standard path structure, and leave it in if the URL differs. This would prevent the need for a whitelist, and instead only a simple regexp or simple parser, but allow for using the registry in the user's configuration as necessary for conforming registries. The interim solution will be to use sed -i 's#http://private-registry/repository/npm#https://registry.yarnpkg.com#' yarn.lock(from @victornoel in #2566 ) (cc @arcanis @BYK @doxiaodong @victornoel ) |
npm is agnostic about which registry you used to generate the package-lock.json since 5.0.0 shipped with Node.js 8.0 [0]. And LeanEngine only uses `package-lock.json` for Node.js 8.0+. But yarn still depends on the registry host in yarn.lock [1]. [0]: https://stackoverflow.com/questions/53814967/what-is-the-point-of-having-resolved-url-in-package-lock-json [1]: yarnpkg/yarn#5892
…680) #680 Currently, when a package is installing its dependencies and those dependencies are in different subtrees of the monorepo with different yarn registries, the update and sync methods in jazelle are not aware of the difference in registries when it stitches the lockfile back together. During the stitching process, jazelle will index all of the lockfiles and then as it builds the lockfiles back up, it will take the highest versioned package across all deps and use that which is correct. However, if one of the projects was resolved with a different package registry, jazelle will potentially overwrite it indiscriminately since it has no concept of ensuring that the registry is correct for that package. This fix uses liberal use of yarn config get registry before installing missing and transitive deps so that the correct registry for the package will be used. In the case of lockfile syncing, this fix will convert all lockfile resolved paths to a relative path instead to be used as the index. Then when lockfiles are rebuilt, the registry that is local to the package (again using yarn config get registry) will be prepended to the relative path to reconstruct the lockfile. There is an open issue in the yarn repo that discusses changing the paths for good to relative paths but there seems to be no recent movement on this: yarnpkg/yarn#5892 The end result of this PR is that: multiple projects in the monorepo can have their own .yarnrc files that point to different registries jazelle will ensure all dependencies are synced across all projects it controls while respecting the configured registry for each project Co-authored-by: Derek Ju <[email protected]> Co-authored-by: UberOpenSourceBot <[email protected]>
|
I (Redhat productization team) recently created a new npm library called lock-treatment-tool https://www.npmjs.com/package/lock-treatment-tool in order to treat lock files before initialising or installing the npm project. It will treat either package-lock.json/yarn.lock files, remove/replace the registries and integrity hash and then you can use your own registries. |
|
I can't believe this issue has existed for 3 years. (#3330) |
|
@PMExtra This is the wrong repo for 2.0. The 2.0 repo is here: https://github.com/yarnpkg/berry |
|
@PMExtra anyway you have the chance to use this lock-treatment-tool library https://www.npmjs.com/package/lock-treatment-tool |
|
Thanks @Daniel15 @Ginxo . |
|
@PMExtra so lock-treatment-tool could be a good solution for you |
…(#466) https://github.com/uber/fusionjs/pull/466 Currently, when a package is installing its dependencies and those dependencies are in different subtrees of the monorepo with different yarn registries, the update and sync methods in jazelle are not aware of the difference in registries when it stitches the lockfile back together. During the stitching process, jazelle will index all of the lockfiles and then as it builds the lockfiles back up, it will take the highest versioned package across all deps and use that which is correct. However, if one of the projects was resolved with a different package registry, jazelle will potentially overwrite it indiscriminately since it has no concept of ensuring that the registry is correct for that package. This fix uses liberal use of yarn config get registry before installing missing and transitive deps so that the correct registry for the package will be used. In the case of lockfile syncing, this fix will convert all lockfile resolved paths to a relative path instead to be used as the index. Then when lockfiles are rebuilt, the registry that is local to the package (again using yarn config get registry) will be prepended to the relative path to reconstruct the lockfile. There is an open issue in the yarn repo that discusses changing the paths for good to relative paths but there seems to be no recent movement on this: yarnpkg/yarn#5892 The end result of this PR is that: multiple projects in the monorepo can have their own .yarnrc files that point to different registries jazelle will ensure all dependencies are synced across all projects it controls while respecting the configured registry for each project Co-authored-by: Derek Ju <[email protected]> Co-authored-by: UberOpenSourceBot <[email protected]>
…(#466) https://github.com/uber/fusionjs/pull/466 Currently, when a package is installing its dependencies and those dependencies are in different subtrees of the monorepo with different yarn registries, the update and sync methods in jazelle are not aware of the difference in registries when it stitches the lockfile back together. During the stitching process, jazelle will index all of the lockfiles and then as it builds the lockfiles back up, it will take the highest versioned package across all deps and use that which is correct. However, if one of the projects was resolved with a different package registry, jazelle will potentially overwrite it indiscriminately since it has no concept of ensuring that the registry is correct for that package. This fix uses liberal use of yarn config get registry before installing missing and transitive deps so that the correct registry for the package will be used. In the case of lockfile syncing, this fix will convert all lockfile resolved paths to a relative path instead to be used as the index. Then when lockfiles are rebuilt, the registry that is local to the package (again using yarn config get registry) will be prepended to the relative path to reconstruct the lockfile. There is an open issue in the yarn repo that discusses changing the paths for good to relative paths but there seems to be no recent movement on this: yarnpkg/yarn#5892 The end result of this PR is that: multiple projects in the monorepo can have their own .yarnrc files that point to different registries jazelle will ensure all dependencies are synced across all projects it controls while respecting the configured registry for each project Co-authored-by: Derek Ju <[email protected]> Co-authored-by: UberOpenSourceBot <[email protected]>
|
👀 some engineers use local registry proxies, like Nexus to test locally, improve downloads, and to work offline (assuming packages are already downloaded). I tried passing |
|
This issue is fixed in Yarn 2+, and the fix will not be backported in 1.x. Please check the Migration guide and continue from there. |
and others
We want to remove hostnames from the lockfiles:
Would become:
This will allow us to switch the default registry more easily if we need to (for example if we want to deprecate our mirror, or if npm suddenly disappear from the surface of the earth), and will also make it easier for the users to switch from a registry to another.
The text was updated successfully, but these errors were encountered: