Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --ignoreDependencies to package command #439

Closed
onilton opened this issue Apr 10, 2020 · 24 comments
Closed

Add --ignoreDependencies to package command #439

onilton opened this issue Apr 10, 2020 · 24 comments
Assignees
Labels
feature-request Request for new features or functionality released
Milestone

Comments

@onilton
Copy link

onilton commented Apr 10, 2020

For some extension projects, people may use a bundle file instead of the regular node_modules.

Even if we do that, we may get some errors, when running

npm list --production --parseable --depth=99999 --link=true

Since vsce always tries to list/find dependencies as can be seen in issue #246 .

The workaround suggest is to use --yarn, since it will check for a yarn.lock file, and if it don't find, nothing happens and vsce doesn't run npm list command.

Adding node_modules to ignore file won't help because this happens after we already tried to run npm list

I propose here to add an option to package called --ignoreDependencies, so we don't run the getNpmDependencies and instead just return [cwd] in getDependencies

@joaomoreno
Copy link
Member

Can you show me what a bundle file means?

@joaomoreno joaomoreno added the info-needed Issue requires more information from poster label Apr 14, 2020
@joaomoreno joaomoreno self-assigned this Apr 14, 2020
@onilton
Copy link
Author

onilton commented Apr 17, 2020

Like in here:

https://code.visualstudio.com/api/working-with-extensions/bundling-extension#configure-webpack

 output: {
    // the bundle is stored in the 'dist' folder (check package.json), 📖 -> https://webpack.js.org/configuration/output/
    path: path.resolve(__dirname, 'dist'),
    filename: 'extension.js',
    libraryTarget: 'commonjs2',
    devtoolModuleFilenameTemplate: '../[resource-path]'
  },

The only thing that needs to be packaged is extension.js (bundle with all the required libs).

But vsce still tries to look at node_modules even if it is not necessary (and sometimes this can cause issues).

@joaomoreno
Copy link
Member

As mentioned here, once you start bundling, you should simply ignore node_modules using the .vscodeignore file.

@onilton
Copy link
Author

onilton commented Apr 27, 2020

@joaomoreno That does not solve the issue. I already have node_modules in my .vscodeignore.

Even if node_modules is included in .vscodeignore , npm list is still run. That's the problem.

Check the following files:

return getDependencies(cwd, useYarn, dependencyEntryPoints).then(deps => {

export function getDependencies(cwd: string, useYarn = false, packagedDependencies?: string[]): Promise<string[]> {

.then(() => exec('npm list --production --parseable --depth=99999 --loglevel=error', { cwd, maxBuffer: 5000 * 1024 }))

@joaomoreno
Copy link
Member

Ah the problem is that it still runs, and it throws an error? What error does it throw, btw?

@joaomoreno joaomoreno reopened this Apr 27, 2020
@onilton
Copy link
Author

onilton commented Apr 27, 2020

It doesn't necessarily always show an error. But in some scenarios it can show an error like the one described in the issue #246 . (the workaround proposed is to use --yarn, that doesn't use npm list.

My error looks like the one in the issue, but it is not the same one (although it is also caused by npm list). Unfortunately I can't post my exact error right now (I am not working in the project that have the issue).

My packaging is working perfectly, but npm list show an error for some dependency that is not really important and that forbids packaging.

Actually, I've never even run npm list before I found this issue.

@joaomoreno
Copy link
Member

So... npm install works without an issue, yet npm list fails? It would be great to see the error. Also, which npm version are you on?

@yhatt

This comment has been minimized.

@onilton
Copy link
Author

onilton commented May 4, 2020

Npm version, 6.13.4, node version v12.16.1

@onilton
Copy link
Author

onilton commented May 4, 2020

npm install works without an issue.

bundle.

npm run package (node ./node_modules/vsce/out/vsce package")

Command failed: npm list --production --parseable --depth=99999 --loglevel=error
npm ERR! peer dep missing: @material-ui/core@^3.2.0, required by [email protected]
npm ERR! peer dep missing: @material-ui/icons@^3.0.1, required by [email protected]

@pmahend1
Copy link

pmahend1 commented Aug 9, 2020

Same issue

 ERROR  Command failed: npm list --production --parseable --depth=99999 --loglevel=error
npm ERR! extraneous: [email protected] C:\Source\GitHub\resxpress\node_modules\make-fetch-happen\node_modules\p-map
npm ERR! missing: p-map@^4.0.0, required by [email protected]

npm run webpack succeeds but this fails

@kube
Copy link

kube commented Aug 29, 2020

I'm having the exact same issue when using a Github Action to deploy a VSCode extension:

It's in a monorepo using Lerna, and my VSCode extension package depends on another library in the repository:

- packages/
  - library/
  - vscode-extension/ # (depends on library)

There are 3 main jobs:

  • install (using lerna link, so library is loaded from folder directly)
  • build (using webpack, and get a bundle, so no further need for node_modules after that)
  • publish (using vsce, which fails because node_modules are not present)

In the publish job, I should not need to load the node_modules cache, which even if fast, is not useful in the process.


So IMO this --ignoreDependencies flag would be useful.

@kube
Copy link

kube commented Aug 29, 2020

I think a simple solution here would be to say that if node_modules is present in the .vscodeignore file, there should be no check done about the dependencies.

@YXL76
Copy link

YXL76 commented Sep 27, 2020

I think the flag is useful because I use yarn 2 which has no list command

SReichelt added a commit to SReichelt/slate that referenced this issue Jan 16, 2021
@joechung-msft
Copy link

I was able to work around the issue by downgrading to npm@6 and recreating package-lock.json not to use lockfile version 2. Seems like a problem with the new lockfile version introduced with npm@7.

@onilton
Copy link
Author

onilton commented Mar 2, 2021

@joaomoreno do you still think we need more info? I see the label is still there.

@joaomoreno joaomoreno added bug Issue identified by VS Code Team member as probable bug and removed info-needed Issue requires more information from poster labels Mar 8, 2021
@DumpySquare
Copy link

what information do we need for this? I've run into the same problem. I have an extension that includes other projects that are referenced directly during development, like below.

npm install works fine, but the npm list --production --parseable --depth=99999 --loglevel=error causes problems because it is finding all the dependencies from the other projects as "extraneous"

	"dependencies": {
		"@f5devcentral/f5-fast-core": "^0.7.0",
		"@types/js-yaml": "^3.12.6",
		"axios": "^0.21.1",
		"f5-conx-core": "file:../f5-conx-core",
		"f5-corkscrew": "file:../f5-corkscrew",
		"js-yaml": "^3.14.1",
		"keytar": "^6.0.1"
	}
ted@r2d2:~/projects/vscode-f5$ vsce package ?
Executing prepublish script 'npm run vscode:prepublish'...

> [email protected] vscode:prepublish /home/ted/projects/vscode-f5
> npm run compile


> [email protected] compile /home/ted/projects/vscode-f5
> tsc -p ./

 ERROR  Command failed: npm list --production --parseable --depth=99999 --loglevel=error
npm ERR! extraneous: @types/[email protected] /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/@types/assert
npm ERR! extraneous: @types/[email protected] /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/@types/mocha
npm ERR! extraneous: @typescript-eslint/[email protected] /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/@typescript-eslint/eslint-plugin
npm ERR! extraneous: @typescript-eslint/[email protected] /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/@typescript-eslint/parser
npm ERR! extraneous: [email protected] /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/assert
npm ERR! extraneous: [email protected] /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/eslint
npm ERR! extraneous: [email protected] /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/mocha
npm ERR! extraneous: [email protected] /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/nyc
npm ERR! extraneous: [email protected] /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/ts-node
npm ERR! extraneous: [email protected] /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/typescript

*****tried using yarn, but failed*****
ted@r2d2:~/projects/vscode-f5$ vsce package --yarn
Executing prepublish script 'yarn run vscode:prepublish'...
/bin/sh: 1: yarn: not found
 ERROR  yarn failed with exit code 127

*****removed ``"vscode:prepublish.old": "npm run compile",`` from package.json scripts, now yarn works*****
ted@r2d2:~/projects/vscode-f5$ vsce package --yarn
 DONE  Packaged: /home/ted/projects/vscode-f5/vscode-f5-3.0.0-beta-1.vsix (78 files, 5.39MB)
 INFO  
The latest version of vsce is 1.86.0 and you have 1.85.0.
Update it now: npm install -g vsce
ted@r2d2:~/projects/vscode-f5$ 

@Livven
Copy link

Livven commented Oct 22, 2021

@joaomoreno npm list seems to perform much more validation than npm install, not all of which is actually relevant, such as peer dependencies missing or being included in different versions (that still work) and so on. Especially if the JS is bundled these validations are completely unnecessary anyway.

For now using --yarn is effective as a workaround, but a more "semantic" flag to disable the npm list call would be appreciated.

@joaomoreno joaomoreno added feature-request Request for new features or functionality and removed bug Issue identified by VS Code Team member as probable bug labels Nov 2, 2021
@joaomoreno joaomoreno added this to the October 2021 milestone Nov 2, 2021
@joaomoreno
Copy link
Member

Will add a --no-dependencies flag.

@github-actions
Copy link

github-actions bot commented Nov 2, 2021

🎉 This issue has been resolved in version 1.102.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@YXL76 YXL76 mentioned this issue Nov 18, 2021
mkizka added a commit to mkizka/blogview that referenced this issue Jan 13, 2022
@pkit
Copy link

pkit commented Feb 12, 2022

@joaomoreno please update the docs about bundling here
As these must be run with --no-dependencies for bundling to work.
Otherwise either node_modules will still be included and it will still complain about bundling size:

This extension consists of 1614 files, out of which 1029 are JavaScript files. 
For performance reasons, you should bundle your extension: https://aka.ms/vscode-bundle-extension . 
You should also exclude unnecessary files by adding them to your .vscodeignore: https://aka.ms/vscode-vscodeignore

Or it will just die with the error from #439 (comment)

@ssbarnea
Copy link
Contributor

ssbarnea commented Jun 16, 2022

I think that when this bug was fixed the ability to load the dependencies from package.json file did not work.

Based on REAME.md file, we were supposed to be able to configure inside package.json:

{
  ...
  "vsce": {
    "dependencies": false,
    "useYarn": false
  }
}

While useYarn works as expected and allow you to avoid having to add --no-yarn, the dependencies option has not effect at all, preventing you from fixing the package command from inside config.

That is indeed a big deal if you do not call vsce package yourself and not having ability to add the required extra arguments.

I tried to look at the d797b8d to spot the bug but I was not able to find exact what was missed.

@joaomoreno If you could double check this and maybe give a hint on why setting this option in package.json does not work it would be awesome. I was hit by this while trying to find a workaround for redhat-developer/vscode-extension-tester#451

@joaomoreno
Copy link
Member

@ssbarnea Do you mind creating an issue explaining what exactly the bug is?

@ssbarnea
Copy link
Contributor

Raise at #742 and PR will appear soon.

smithgp added a commit to forcedotcom/analyticsdx-vscode that referenced this issue Jun 29, 2022
This is to work around several issues use npm 7 or 8 with vscode
packaging:
* microsoft/vscode-vsce#439
* microsoft/vscode-vsce#673
smithgp added a commit to forcedotcom/analyticsdx-vscode that referenced this issue Jun 29, 2022
This is to work around several issues use npm 7 or 8 with vscode
packaging:
* microsoft/vscode-vsce#439
* microsoft/vscode-vsce#673
smithgp added a commit to forcedotcom/analyticsdx-vscode that referenced this issue Jun 30, 2022
* Update to node 16.13.10

* Update to vscode 1.67 api

* Update to typescript 4.6.3

* Update to prettier 2.6.2

* Update shx to 3.3.4

* Update request-light to 0.5.8

* Update glob to 8.0.3

* Update semver to 7.3.7

* Update sinon to 14.0.0

* Update chai to 4.3.6

* Update mocha to 10.0.0

* Update @vscode/test-electron to 2.1.3

* Update better-ajv-erros to 1.2.0

* Update vsce to 1.103.1

* Use npm 6 for build & packaging

This is to work around several issues use npm 7 or 8 with vscode
packaging:
* microsoft/vscode-vsce#439
* microsoft/vscode-vsce#673

* Fix for vscode tests on windows & mac
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request Request for new features or functionality released
Projects
None yet
Development

No branches or pull requests