@Salesforce/cli not installable when npm artifact is not allowed, can it use the system npm instead

[hungrypipo]

Our company releases NodeJS LTS versions with packaged npm version. And block the npm artifact itself. Is it possible to make the two dependencies that require the npm artifact use the system level npm that comes with NodeJS instead of pulling the npm artifact as a dependency?

[github-actions[bot]]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[github-actions[bot]]

Hello @hungrypipo 👋 It looks like you didn't include the full Salesforce CLI version information in your issue.
Please provide the output of version --verbose --json for the CLI you're using (sf or sfdx).

A few more things to check:

• Make sure you've provided detailed steps to reproduce your issue.
  • A repository that clearly demonstrates the bug is ideal.
• Make sure you've installed the latest version of Salesforce CLI. (docs)
  • Better yet, try the rc or nightly versions. (docs)
• Try running the doctor command to diagnose common issues.
• Search GitHub for existing related issues.

Thank you!

[iowillhoit]

Hello @hungrypipo, did the fix that we implemented for you not work?

#2327
oclif/plugin-plugins#647
salesforcecli/plugin-trust#572

FWIW: We used to use the system npm but it would fail when npm was not installed. We would also sometimes see odd/difficult to debug issues related to a specific user installed version of npm. For reliability and confidence through testing, we started to bundle npm (oclif/plugin-plugins#595)

We may explore this (if for no other reason than reducing our bundle size) but it is not on our immediate roadmap.

[hungrypipo]

Hi @iowillhoit nice to chat with you again. Unfortunately I thought matching the npm to the NodeJS would be enough but our company keeps the npm artifact out of our artifact repository(it's blocked) we don't hit the npmjs repo.
My test worked on an outside our company test machine, I didn't realize right away that the install was pulling the npm artifact and installing it locally to the cli, I mistakenly thought it was using the NodeJS LTS one since it was the same version.

Now that the npm matches the NodeJS LTS we don't have to do overrides if we install local instead of global, I know you mentioned installing as local in the previous issue, wondering what the side effects of that are? Does it still work with VSCode? Do I just need to add it to the path?

[hungrypipo]

@iowillhoit for the npm version in the two plugins I notice it's set to a specific version and not to a specific version and higher. Just wondering if you could make that tweak? Because if we're at a NodeJS LTS version and they push an update to the next NodeJS LTS it will force us to update the CLI right away. We use the CLI in Jenkins pipelines so it installs the CLI on each run of the pipeline, those would break till we procured the next CLI version into our artifact repo.

[iowillhoit]

Hey @hungrypipo, I think as long as your locally installed sf is on your $PATH the VSCode should be able to find it.

I am not sure I am following your second comment. Even if we added a ^ to the npm deps, they install would still install from your internal registry (which you said has npm blocked).

FWIW, I did discuss this with the team. We created an exploratory ticket to look into how feasible it would be to optionally install npm based on what the user already has installed in their environment. However, I am not sure when it will get picked up or if we will decide to move forward after we explore it a bit. (W-14496387)

[louis-bompart]

@iowillhoit 👋 might be slightly related, but installing the Salesforce CLI and then running npm update, for example like so:

mkdir temp-project
cd temp-project
npm init -y 
npm i @salesforce/cli
npm update

Will make Arborist choke up:

npm ERR! code ERR_INVALID_ARG_TYPE
npm ERR! The "from" argument must be of type string. Received undefined

Here's the JS stack associated:

TypeError [ERR_INVALID_ARG_TYPE]: The "from" argument must be of type string. Received undefined
    at new NodeError (node:internal/errors:405:5)
    at validateString (node:internal/validators:162:11)
    at relative (node:path:497:5)
    at ~\npm_global_path\node_modules\npm\node_modules\@npmcli\arborist\lib\arborist\reify.js:1132:21
    at Array.map (<anonymous>)
    at ~\npm_global_path\node_modules\npm\node_modules\@npmcli\arborist\lib\arborist\reify.js:1130:66
    at Array.map (<anonymous>)
    at [rollbackMoveBackRetiredUnchanged] (~\npm_global_path\node_modules\npm\node_modules\@npmcli\arborist\lib\arborist\reify.js:1130:8)
    at [reifyPackages] (~\npm_global_path\node_modules\npm\node_modules\@npmcli\arborist\lib\arborist\reify.js:256:31)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Most NPM commands that need to reify the actual tree are likely to fail :/
Tell me if you prefer I open a separate issue for this.

To rule out my env, I run the exact script mentioned above in a GHA, and see the same issue: https://github.com/louis-bompart/shiny-disco

[iowillhoit]

Hey @louis-bompart, that is a different issue than what were discussing here. At a glance I am not sure what is going on there. However, we don't recommend installing sf as a dependency in a project. Try installing it globally instead (docs). FWIW, I just ran npm update --global and it updated my globally installed sf version without issue.

[gfarb]

@iowillhoit For CI/CD processes we need to have sf as a project dependency, but now we are starting to run into this as well

[erocheleau]

@iowillhoit I agree with @gfarb, sf cli is a project dependency especially for CI/CD, it's also a dev dependency for development, so in theory it should be in the dev dependencies.

The problems with having it globally are the following:

1. Needs a separate setup step in order to work with a project to install sf-cli since it's not part of the npm i local to the project. Not a good practice.
2. Needs a separate step in any CI pipeline but also in any CD tasks to create a package version/release a package/etc.
3. Cannot easily "fix" a version which leads to periodic issues when versions of the CLI get updated with potential bugs (even if you fix them quickly)
4. We lose all tooling around scanning for vulnerable dependencies since sf-cli is not in the package-lock.

[hungrypipo]

I disagree that SF-cli is a project dependency it’s CD/CI tooling. It’s literally the cli that gets your code from repo to target org. It’s in the same category as Jenkins or any other CD/CI tool.
Project dependency to me is something the code in the project is directly using say like a crypto library.

[erocheleau]

I'm saying dev dependency, I agree with your definition of project dependency (we don't bundle/ship the sf cli with out project).

But not to be pedantic or anything but this is the literal definition of a devDependency according to npm:

"devDependencies": Packages that are only needed for local development and testing.

You use GitHub actions for example as your CI/CD runner, but all the other tooling you need to develop locally and to test are dev dependencies. I can't develop or test my code without using sf cli (which is amazing btw), but so it should fall in the category of dev dependencies.

The Jenkins example doesn't really fit, as it's not the same thing as the Salesforce CLI. It's more like gulp for example.

Especially once you add to the mix that you need to control the different versions of the cli in order to do different things, for example its version can add support for a newer "feature", "object" or "syntax" supported by Salesforce. You need to know on which version of the CLI you are as much as which version of other tools you are using.

[hungrypipo]

Hey @hungrypipo, I think as long as your locally installed sf is on your $PATH the VSCode should be able to find it.

I am not sure I am following your second comment. Even if we added a ^ to the npm deps, they install would still install from your internal registry (which you said has npm blocked).

FWIW, I did discuss this with the team. We created an exploratory ticket to look into how feasible it would be to optionally install npm based on what the user already has installed in their environment. However, I am not sure when it will get picked up or if we will decide to move forward after we explore it a bit. (W-14496387)

@iowillhoit The reason for this, is because they will update NodeJS on our machines which will update the npm version, so when that happens I don't want to not able to be able to install the specific sf-cli we're using. The sf-cli installs on each Jenkins run.