Autocomplete bug with yarn run

[7rulnik]

Autocomplete for yarn looks like this:

I checked a few commits. Seems that bug was introduced in a47b58e#diff-7d8d4a411c6bd5f832a76535275f89e1

Is it related to the line-break symbol?

[7rulnik]

/cc @MaximDevoir

[MaximDevoir]

Thanks for reporting this. What does your scripts object from package.json look like.

[7rulnik]

That's enough to reproduce it in my case

{
    "scripts": {
        "lint": "yarn pretty && yarn eslint",
        "foo": "blabla"
    }
}

And even without script section I get concatenated deps. So I believe package.json is not problem here.

I'm using [email protected] and macOS 10.15 (19A583).

There is result for yarn run --json:

{"type":"info","data":"Commands available from binary scripts: acorn, ansi-html, atob, browserslist, cssesc, detect-libc, errno, eslint, eslint-config-prettier-check, esparse, esvalidate, he, html-minifier, import-local-fixture, js-yaml, jsesc, json5, leasot, leasot-reporter, loose-envify, miller-rabin, mime, mkdirp, multicast-dns, needle, node-pre-gyp, nopt, parser, prettier, rc, regexp-tree, regjsparser, rimraf, semver, sha.js, tcm, terser, tsc, tsserver, uglifyjs, uuid, webpack, webpack-cli, webpack-dev-server, which, yarn-deduplicate"}
{"type":"error","data":"There are no scripts specified inside package.json."}

And I got this as a result of sed:

sed -E '/Commands available/!d;s/.*Commands available from binary scripts: ([^"]+)".*/\1/;s/.*"items":\[([^]]+).*/\1/;s/[" ]//g;s/:/\\:/g;s/,/\n/g' <<< `yarn run --json 2>/dev/null`
acornnansi-htmlnatobnbrowserslistncssescndetect-libcnerrnoneslintneslint-config-prettier-checknesparsenesvalidatenhenhtml-minifiernimport-local-fixturenjs-yamlnjsescnjson5nleasotnleasot-reporternloose-envifynmiller-rabinnmimenmkdirpnmulticast-dnsnneedlennode-pre-gypnnoptnparsernprettiernrcnregexp-treenregjsparsernrimrafnsemvernsha.jsntcmnterserntscntsservernuglifyjsnuuidnwebpacknwebpack-clinwebpack-dev-servernwhichnyarn-deduplicate

[MaximDevoir]

Your yarn run --json command couldn't find any scripts inside package.json. I don't think that's just a coincidence. yarn run --json should include a line like this:

{"type":"list","data":{"type":"possibleCommands","items":["foo","lint"],"hints":{"foo":"blabla","lint":"yarn pretty && yarn eslint"}}}

Yarn uses a custom readJson function to parse your package.json file.

ZSH completions depends on yarn run --json and Yarn are failing to parse your part of package.json, specifically the scripts object. This might be an encoding issue with your package.json.

I don't have access to a Mac and I don't think this is a line-endings issue. Although, it's not unheard of for Yarn to mangle with line-endings (e.g: mixing \n and \r and \r\n together. see yarnpkg/yarn#5480 or pattern-lab/patternlab-node#967). Even with mangled line endings, the yarn completions work on my machine (Ubuntu 18.04).

[MaximDevoir]

Could you upload a file of the package.json your were able to reproduce this issue with from your comment here.

What do you get when you run the old line for reading binaries/scripts:

yarn run --json 2>/dev/null | sed -E '/Commands available|possibleCommands/!d;s/.*Commands available from binary scripts: ([^"]+)".*/\1/;s/.*"items":\[([^]]+).*/\1/;s/[" ]//g' | tr , '\n' | sed -e 's/:/\\:/g'

Edit: An older version of this comment included the wrong script to run. This has been updated with the correct script to run.

[7rulnik]

So, that's my package.json:

{
    "name": "blabla",
    "version": "0.1.0",
    "private": true,
    "main": "index.js",
    "scripts": {
        "test": "echo \"Error: no test specified\" && exit 1",
        "start": "webpack-dev-server --mode development --watch --hot --progress --color --port 3000",
        "build": "rm -rf 'dist/*' && webpack-cli --mode production --progress --color",
        "tcm": "tcm src -c && yarn pretty -- --fix",
        "eslint": "eslint --max-warnings 200 \"src/**/*.{ts,tsx}\"",
        "format-check": "prettier --check \"src/**/*.{ts,tsx}\"",
        "format-fix": "yarn eslint --fix",
        "pretty": "prettier --write \"src/**/*.{ts,tsx}\"",
        "lint": "yarn pretty && yarn eslint"
    },
    "license": "MIT",
    "devDependencies": {
        "@babel/core": "^7.2.2",
        "@babel/plugin-proposal-class-properties": "^7.3.4",
        "@babel/plugin-proposal-object-rest-spread": "^7.6.2",
        "@babel/preset-env": "^7.3.4",
        "@babel/preset-react": "^7.0.0",
        "@types/history": "^4.7.2",
        "@types/lodash": "^4.14.144",
        "@types/node": "^10.12.18",
        "@types/react": "^16.9.2",
        "@types/react-dom": "^16.9.0",
        "@types/react-redux": "^7.1.2",
        "@types/react-router": "^4.4.3",
        "@types/react-router-dom": "^4.3.1",
        "@types/styled-components": "^4.1.19",
        "@types/webpack": "^4.4.23",
        "@types/webpack-env": "1.13.6",
        "@types/yup": "^0.26.24",
        "@typescript-eslint/eslint-plugin": "^2.3.3",
        "@typescript-eslint/parser": "^2.3.3",
        "babel-loader": "^8.0.5",
        "babel-plugin-transform-react-remove-prop-types": "^0.4.24",
        "babel-plugin-transform-remove-console": "^6.9.4",
        "css-loader": "^2.1.1",
        "eslint": "^5.15.1",
        "eslint-config-airbnb": "^17.1.0",
        "eslint-config-prettier": "^4.1.0",
        "eslint-loader": "^2.1.2",
        "eslint-plugin-array-func": "^3.1.3",
        "eslint-plugin-filenames": "^1.3.2",
        "eslint-plugin-import": "^2.16.0",
        "eslint-plugin-jest": "^22.3.0",
        "eslint-plugin-jsx-a11y": "^6.2.1",
        "eslint-plugin-new-with-error": "^2.0.0",
        "eslint-plugin-optimize-regex": "^1.1.6",
        "eslint-plugin-prettier": "^3.0.1",
        "eslint-plugin-promise": "^4.0.1",
        "eslint-plugin-react": "^7.12.4",
        "eslint-plugin-sonarjs": "^0.3.0",
        "eslint-plugin-sort-destructure-keys": "^1.2.0",
        "file-loader": "^3.0.1",
        "fork-ts-checker-webpack-plugin": "^1.5.0",
        "html-webpack-plugin": "^3.2.0",
        "monaco-editor-webpack-plugin": "^1.7.0",
        "postcss-loader": "^3.0.0",
        "prettier": "^1.16.4",
        "prettier-loader": "^2.1.1",
        "react-hot-loader": "^4.6.3",
        "style-loader": "^0.23.1",
        "styled-components": "^4.1.3",
        "todo-webpack-plugin": "^1.9.9",
        "typed-css-modules": "^0.4.2",
        "typescript": "^3.4.5",
        "url-loader": "^1.1.2",
        "webpack": "^4.28.4",
        "webpack-cleanup-plugin": "^0.5.1",
        "webpack-cli": "^3.2.1",
        "webpack-dev-server": "^3.1.14",
        "yarn-deduplicate": "^1.1.1"
    },
    "dependencies": {
        "@alfabank/base-text": "^2.6.0",
        "@atlaskit/table-tree": "^6.0.2",
        "@babel/preset-typescript": "^7.6.0",
        "antd": "^3.23.6",
        "axios": "^0.18.0",
        "classnames": "^2.2.6",
        "evergreen-ui": "^4.14.0",
        "final-form": "^4.18.5",
        "history": "^4.7.2",
        "lodash": "^4.17.15",
        "monaco-editor": "0.17.1",
        "react": "^16.9.0",
        "react-dom": "^16.9.0",
        "react-final-form": "^6.3.0",
        "react-intl": "^2.8.0",
        "react-monaco-editor": "^0.28.0",
        "react-redux": "^7.1.1",
        "react-router": "^4.3.1",
        "react-router-dom": "^4.3.1",
        "redux-starter-kit": "^0.7.0",
        "yup": "^0.27.0"
    },
    "engines": {
        "yarn": ">=1.12",
        "node": ">=10.15",
        "npm": ">=6.4.1"
    }
}

That's yarn run --json result:

{"type":"info","data":"Commands available from binary scripts: acorn, ansi-html, atob, browserslist, cssesc, detect-libc, errno, eslint, eslint-config-prettier-check, esparse, esvalidate, he, html-minifier, import-local-fixture, js-yaml, jsesc, json5, leasot, leasot-reporter, loose-envify, miller-rabin, mime, mkdirp, multicast-dns, needle, node-pre-gyp, nopt, parser, prettier, rc, regexp-tree, regjsparser, rimraf, semver, sha.js, tcm, terser, tsc, tsserver, uglifyjs, uuid, webpack, webpack-cli, webpack-dev-server, which, yarn-deduplicate"}
{"type":"info","data":"Project commands"}
{"type":"list","data":{"type":"possibleCommands","items":["build","eslint","format-check","format-fix","lint","pretty","start","tcm","test"],"hints":{"build":"rm -rf 'dist/*' && webpack-cli --mode production --progress --color","eslint":"eslint --max-warnings 200 \"src/**/*.{ts,tsx}\"","format-check":"prettier --check \"src/**/*.{ts,tsx}\"","format-fix":"yarn eslint --fix","lint":"yarn pretty && yarn eslint","pretty":"prettier --write \"src/**/*.{ts,tsx}\"","start":"webpack-dev-server --mode development --watch --hot --progress --color --port 3000","tcm":"tcm src -c && yarn pretty -- --fix","test":"echo \"Error: no test specified\" && exit 1"}}}
{"type":"error","data":"No command specified."}

And result for old line:

yarn run --json 2>/dev/null | sed -E '/Commands available|possibleCommands/!d;s/.*Commands available from binary scripts: ([^"]+)".*/\1/;s/.*"items":\[([^]]+).*/\1/;s/[" ]//g' | tr , '\n' | sed -e 's/:/\\:/g'

acorn
ansi-html
atob
browserslist
cssesc
detect-libc
errno
eslint
eslint-config-prettier-check
esparse
esvalidate
he
html-minifier
import-local-fixture
js-yaml
jsesc
json5
leasot
leasot-reporter
loose-envify
miller-rabin
mime
mkdirp
multicast-dns
needle
node-pre-gyp
nopt
parser
prettier
rc
regexp-tree
regjsparser
rimraf
semver
sha.js
tcm
terser
tsc
tsserver
uglifyjs
uuid
webpack
webpack-cli
webpack-dev-server
which
yarn-deduplicate
build
eslint
format-check
format-fix
lint
pretty
start
tcm
test

[7rulnik]

@MaximDevoir any thoughts?

[MaximDevoir]

It looks like Mac OS doesn't process newlines from sed (see here and here). I'll try to submit a fix for this today.

[MaximDevoir]

Fixed in #666

[iulo]

@MaximDevoir seems like this fix still not merge into master

[MaximDevoir]

Yes, we need someone from @zsh-users to merge #666 and #668 .

What I have done, since I needed these fixes for my machine, is to add the changes from those PRs to my local oh-my-zsh Yarn plugin file.

diff --git a/src/_yarn b/src/_yarn
index 3689ae96..7cc8051b 100644
--- a/src/_yarn
+++ b/src/_yarn
@@ -86,9 +86,11 @@ _yarn_scripts() {
   local i runJSON
 
   runJSON=$(yarn run --json 2>/dev/null)
-  binaries=($(sed -E '/Commands available/!d;s/.*Commands available from binary scripts: ([^"]+)".*/\1/;s/.*"items":\[([^]]+).*/\1/;s/[" ]//g;s/:/\\:/g;s/,/\n/g' <<< "$runJSON"))
-  scriptNames=($(sed -E '/possibleCommands/!d;s/.*"items":\[([^]]+).*/\1/;s/[" ]//g;s/:/\\:/g;s/,/\n/g' <<< "$runJSON"))
-  scriptCommands=("${(@f)$(sed -E '/possibleCommands/!d;s/.*"hints":\{([^}]+)\}.*/\1/;s/"[^"]+"://g;s/:/\\:/g;s/","/\n/g;s/(^"|"$)//g' <<< "$runJSON")}")
+  # Some sed utilities (e.g. Mac OS / BSD) don't interpret `\n` in a replacement
+  # pattern as a newline. See https://superuser.com/q/307165
+  binaries=($(sed -E '/Commands available/!d;s/.*Commands available from binary scripts: ([^"]+)".*/\1/;s/.*"items":\[([^]]+).*/\1/;s/[" ]//g;s/:/\\:/g;s/,/\'$'\n/g' <<< "$runJSON"))
+  scriptNames=($(sed -E '/possibleCommands/!d;s/.*"items":\[([^]]+).*/\1/;s/[" ]//g;s/:/\\:/g;s/,/\'$'\n/g' <<< "$runJSON"))
+  scriptCommands=("${(@f)$(sed -E '/possibleCommands/!d;s/.*"hints":\{(.+")\}.*/\1/;s/"[^"]+"://g;s/:/\\:/g;s/","/\'$'\n/g;s/(^"|"$)//g' <<< "$runJSON")}")
 
   for (( i=1; i <= $#scriptNames; i++ )); do
     scripts+=("${scriptNames[$i]}:${scriptCommands[$i]}")

Then, once the PRs are merged, I will go back to using the origin master branch in ~/.oh-my-zsh

[MaximDevoir]

Pinging @mtorromeo and @nicoulaj. PRs #668 and #666 are ready for merging.

[7rulnik]

@mtorromeo @nicoulaj sorry guys for annoying, but it's the blocker issue for people who are using macOS