yarn install --production misses some required transitive dependencies

[chrisdarroch]

Running yarn install works fine. yarn install --production should also work, installing only those dependencies needed outside of dev, but it misses some transitive dependencies that are present in the devDependencies section.

Steps to reproduce

1. Create a new project, using this package.json:

{
  "name": "yarndemo",
  "dependencies": {
    "gulp": "3.9.1",
    "gulp-help": "1.6.1",
    "maven": "^4.0.3",
    "run-sequence": "1.2.2",
    "xml2js": "^0.4.17",
    "yargs": "5.0.0"
  },
  "devDependencies": {
    "eslint": ">=3.3.0",
    "eslint-config-airbnb": "0.1.0",
    "eslint-plugin-react": "3.5.1",
    "stylelint": "^7"
  },
  "engines": {
    "node": ">=6.7.0",
    "npm": "3.10.7"
  },
  "scripts": {
    "postinstall": "npm list --production"
  },
  "private": true
}

2. Ensure no node_modules directory exists.
3. Delete yarn.lock if it already exists.
4. Run yarn install --production.

Expected behavior

Running npm list --production (as done in the postinstall script) should report no missing dependencies.

Actual behaviour

Several transitive dependencies are missing. e.g., error-ex@^1.2.0, [email protected], and others.

Notes

This behaviour is consistent regardless of whether the lockfile is present before running the command.

Please mention your node.js, yarn and operating system version.

Using Yarn 0.18.0, Node.js v6.7.0, OSX 10.12.1 (Darwin Kernel Version 16.1.0: Thu Oct 13 21:26:57 PDT 2016; root:xnu-3789.21.3~60/RELEASE_X86_64), and zsh.

[bestander]

Could you check in yarn 0.18.1?

[chrisdarroch]

It's much closer to being correct, but still fails on one of the transitive dependencies:

yarn install v0.18.1
info No lockfile found.
[1/4] 🔍  Resolving packages...
warning gulp > vinyl-fs > glob-stream > [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
warning gulp > vinyl-fs > glob-watcher > gaze > globule > [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
warning gulp > vinyl-fs > glob-watcher > gaze > globule > glob > [email protected]: graceful-fs v3.0.0 and before will fail on node releases >= v7.0. Please update to graceful-fs@^4.0.0 as soon as possible. Use 'npm ls graceful-fs' to find it in the tree.
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
$ npm list --production
(snip...)
├─┬ [email protected]
│ └─┬ [email protected]
│   ├─┬ [email protected]
│   │ ├─┬ [email protected]
│   │ │ └─┬ [email protected]
│   │ │   ├── UNMET DEPENDENCY balanced-match@^0.4.1

npm ERR! missing: balanced-match@^0.4.1, required by [email protected]
error Command failed with exit code 1.

[bestander]

Indeed, I can reproduce, looks like we are missing this path for the example package.json:

gulp->vinyl-fs->minimatch->brace-expansion->balanced-match

[migueloller]

Here's a discussion that is relevant to this issue: heroku/heroku-buildpack-nodejs#337

And if it helps, here's a diff of what got installed after running yarn --production and then running npm install --production:

EDIT: #761 seems to be relevant as well.

[sontek]

This seems related to:

#2141

I've been pinging them to re-open but haven't gotten a response yet.

[bestander]

Reopened #2141.
The best thing you can do to speed up the fix is send a PR with a failing test similar to #1754

[bestander]

The minimum reproducible package.json is:

{
  "name": "yarndemo",
  "dependencies": {
    "gulp": "3.9.1"
  },
  "devDependencies": {
    "eslint": ">=3.3.0",
    "eslint-plugin-react": "3.5.1",
    "stylelint": "^7"
  }
}

New check --commonjs can catch it now, I`ll add tests and proceed with the fix

[bestander]

I can still repro the above issue in latest master:

bestander-pro:2263 bestander$ yarn install --production
yarn install v0.20.0-0
info No lockfile found.
warning yarndemo: No license field
[1/4] 🔍  Resolving packages...
warning gulp > vinyl-fs > glob-stream > [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
warning gulp > vinyl-fs > glob-watcher > gaze > globule > [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
warning gulp > vinyl-fs > glob-watcher > gaze > globule > glob > [email protected]: graceful-fs v3.0.0 and before will fail on node releases >= v7.0. Please update to graceful-fs@^4.0.0 as soon as possible. Use 'npm ls graceful-fs' to find it in the tree.
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
✨  Done in 4.25s.


bestander-pro:2263 bestander$ yarn check --production --verify-tree
yarn check v0.20.0-0
error "gulp#vinyl-fs#glob-stream#minimatch#brace-expansion#balanced-match" not installed
error Found 1 errors.
info Visit https://yarnpkg.com/en/docs/cli/check for documentation about this command.

[ebaynaud]

Same issue as #2141 as already said upper. So #2141 should not be closed.

Still an issue with 0.19.1.

And I can provide another very simple package.json and steps to reproduce:

{
  "dependencies": {
    "css-select": "1.2.0"
  },
 "devDependencies": {
    "domutils": "1.5.1"
  }
}

Result:

ebaynaud@McR1 test $ rm -rf node_modules yarn.lock; yarn --prod
yarn install v0.19.1
info No lockfile found.
warning No license field
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
✨  Done in 0.79s.

ebaynaud@McR1 test $ npm ls domutils
/Users/ebaynaud/src/test
└─┬ [email protected]
  └── UNMET DEPENDENCY [email protected]

npm ERR! missing: [email protected], required by [email protected]

Without --prod, everything is ok:

ebaynaud@McR1 test $ rm -rf node_modules yarn.lock; yarn
yarn install v0.19.1
info No lockfile found.
warning No license field
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
✨  Done in 0.84s.

ebaynaud@McR1 test $ npm ls domutils
/Users/ebaynaud/src/test
└── [email protected] 

Ok also using --prod but removing the devDependency in package.json:

{
  "dependencies": {
    "css-select": "1.2.0"
  }
}

Result:

ebaynaud@McR1 test $ rm -rf node_modules yarn.lock; yarn --prod
yarn install v0.19.1
info No lockfile found.
warning No license field
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
✨  Done in 0.91s.

ebaynaud@McR1 test $ npm ls domutils
/Users/ebaynaud/src/test
└─┬ [email protected]
  └── [email protected] 

Hope it can helps to fix that definitely!

[yagudaev]

Also having this issue same as reported by @ebaynaud

├─┬ [email protected]
│ └─┬ [email protected]
│   └─┬ [email protected]
│     ├─┬ [email protected]
│     │ └── UNMET DEPENDENCY [email protected]
│     └─┬ [email protected]
│       └── [email protected]

Seem like this fixes it

$ yarn add [email protected]

But this should not be required :(.

[bestander]

Should be fixed in master now, will be released in 0.20 next week