Issue installing on Windows - Invalid or unexpected token

[eternalmatt]

Hi I get this issue coming from esbuild's install script. This has been happening forever. I have checked #2929 and #1703 and #2812 but couldn't find any similarities despite nearly the exact same error log. My environment does not contain ESBUILD_BINARY_PATH.

I am behind a corporate npm registry and wouldn't be surprised if something got changed in the esbuild package.json somehow.

$ env | grep ESBUILD
(nothing)
$ node -p process.env.ESBUILD_BINARY_PATH 
(nothing)

mkdir esbuild-repro
cd esbuild-repro
npm init -y
npm install esbuild

$ npm i esbuild
npm ERR! code 1
npm ERR! path C:\esbuild-repro\node_modules\esbuild
npm ERR! command failed
npm ERR! command C:\Program Files\Git\bin\bash.exe -c node install.js
npm ERR! node:internal/errors:867
npm ERR!   const err = new Error(message);
npm ERR!               ^
npm ERR!
npm ERR! Error: Command failed: C:\Program Files\nodejs\node.exe C:\esbuild-repro\node_modules\esbuild\bin\esbuild --version
npm ERR! C:\esbuild-repro\node_modules\esbuild\bin\esbuild:1
npm ERR! ELF
npm ERR! ^
npm ERR!
npm ERR! SyntaxError: Invalid or unexpected token
npm ERR!     at internalCompileFunction (node:internal/vm:73:18)
npm ERR!     at wrapSafe (node:internal/modules/cjs/loader:1149:20)
npm ERR!     at Module._compile (node:internal/modules/cjs/loader:1190:27)
npm ERR!     at Module._extensions..js (node:internal/modules/cjs/loader:1280:10)
npm ERR!     at Module.load (node:internal/modules/cjs/loader:1089:32)
npm ERR!     at Module._load (node:internal/modules/cjs/loader:930:12)
npm ERR!     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
npm ERR!     at node:internal/main/run_main_module:23:47
npm ERR!
npm ERR! Node.js v18.14.0
npm ERR!
npm ERR!     at checkExecSyncError (node:child_process:885:11)
npm ERR!     at Object.execFileSync (node:child_process:921:15)
npm ERR!     at validateBinaryVersion (C:\esbuild-repro\node_modules\esbuild\install.js:98:28)
npm ERR!     at C:\esbuild-repro\node_modules\esbuild\install.js:283:5 {
npm ERR!   status: 1,
npm ERR!   signal: null,
npm ERR!   output: [
npm ERR!     null,
npm ERR!     Buffer(0) [Uint8Array] [],
npm ERR!     Buffer(653) [Uint8Array] [
npm ERR!        67,  58,  92, 101, 115,  98, 117, 105, 108, 100,  45, 114,
npm ERR!       101, 112, 114, 111,  92, 110, 111, 100, 101,  95, 109, 111,
npm ERR!       100, 117, 108, 101, 115,  92, 101, 115,  98, 117, 105, 108,
npm ERR!       100,  92,  98, 105, 110,  92, 101, 115,  98, 117, 105, 108,
npm ERR!       100,  58,  49,  13,  10, 127,  69,  76,  70,   2,   1,   1,
npm ERR!        13,  10,  94,  13,  10,  13,  10,  83, 121, 110, 116,  97,
npm ERR!       120,  69, 114, 114, 111, 114,  58,  32,  73, 110, 118,  97,
npm ERR!       108, 105, 100,  32, 111, 114,  32, 117, 110, 101, 120, 112,
npm ERR!       101,  99, 116, 101,
npm ERR!       ... 553 more items
npm ERR!     ]
npm ERR!   ],
npm ERR!   pid: 35300,
npm ERR!   stdout: Buffer(0) [Uint8Array] [],
npm ERR!   stderr: Buffer(653) [Uint8Array] [
npm ERR!      67,  58,  92, 101, 115,  98, 117, 105, 108, 100,  45, 114,
npm ERR!     101, 112, 114, 111,  92, 110, 111, 100, 101,  95, 109, 111,
npm ERR!     100, 117, 108, 101, 115,  92, 101, 115,  98, 117, 105, 108,
npm ERR!     100,  92,  98, 105, 110,  92, 101, 115,  98, 117, 105, 108,
npm ERR!     100,  58,  49,  13,  10, 127,  69,  76,  70,   2,   1,   1,
npm ERR!      13,  10,  94,  13,  10,  13,  10,  83, 121, 110, 116,  97,
npm ERR!     120,  69, 114, 114, 111, 114,  58,  32,  73, 110, 118,  97,
npm ERR!     108, 105, 100,  32, 111, 114,  32, 117, 110, 101, 120, 112,
npm ERR!     101,  99, 116, 101,
npm ERR!     ... 553 more items
npm ERR!   ]
npm ERR! }
npm ERR!
npm ERR! Node.js v18.14.0

If I install with --ignore-scripts, the node_modules looks like this:

$ ls -R node_modules/esbuild/
node_modules/esbuild/:
LICENSE.md  README.md  bin/  install.js  lib/  package.json

node_modules/esbuild/bin:
esbuild

node_modules/esbuild/lib:
main.d.ts  main.js

[evanw]

It looks like esbuild's installer thinks you are installing on Linux (ELF is the executable format for Linux). I notice you are running things via bash.exe. Perhaps that is causing the environment to be detected as Linux instead of Windows? The installer checks node's require('os').platform() function to detect the current OS. In any case, you'll likely need to figure out on your end how to run npm in self-consistent environment. Some ideas (note that I don't use Windows myself, so these are not strong suggestions):

• Use cmd.exe instead of bash to run things using a Windows-native environment.
• Use WSL instead to run things in a real Linux environment.

The way you installed node/npm could hypothetically also affect things. For example, if you installed it using something Linux-related (e.g. WSL), then it might think it's on Linux and/or run things as if it were on Linux. I'm not sure. But that could be something else for you to investigate.

[JSMike]

I've ran into this issue myself recently at work but couldn't replicate on my personal computer. I realized that there appears to be an issue with my company's on-prem npm registry, the node_modules/esbuild/bin/esbuild file was corrupted. If you're experiencing this issue then compare the bin/esbuild file from the public npm with your company's npm and see if you can get library re-uploaded. In my case the bin/esbuild file for our on-prem solution somehow became an actual binary that is 9mb. The one that comes from the public npm registry is only 9kb and is a node/shell script that's plainly readable.

Note: You can use a web cdn to compare to your local https://unpkg.com/[email protected]/bin/esbuild

[JSMike]

It looks like esbuild's installer thinks you are installing on Linux (ELF is the executable format for Linux). I notice you are running things via bash.exe. Perhaps that is causing the environment to be detected as Linux instead of Windows?

That may be part of the case, I noticed that this was only happening on Windows and not darwin/Linux, for some reason I never got this error in a Linux environment, only on Windows. But similar to the OP when installing with --ignore-scripts the bin/esbuild file was an actual binary (similar to the one from @esbuild/win32-x64, assuming it's probably linux version of that) and not a #!/usr/bin/env node script that appears on my home computer when doing a fresh install with --ignore-scripts. So some 'magic' may be happening in the internal npm registry when adding this package for some reason that snapshots the linux file in there.

Note: This was happening for my on-prem npm registry for versions 0.18.x and 0.19.x, the on-prem 0.17.x versions are not corrupted. I was able to pin dependencies to v0.17.x to get successful installs.

[evanw]

the node_modules/esbuild/bin/esbuild file was corrupted. If you're experiencing this issue then compare the bin/esbuild file from the public npm with your company's npm and see if you can get library re-uploaded. In my case the bin/esbuild file for our on-prem solution somehow became an actual binary that is 9mb.

This is a deliberate optimization and is how the installer normally works (if --ignore-scripts is not present, if you aren't using Yarn, and if you're not on Windows). The source code for this is here if you're curious (including a comment about when and why it happens). That does not indicate that the package has been "corrupted". As you noted, it should not happen with --ignore-scripts because of course the install script should not be run. If the install script is still being run with --ignore-scripts then there's some bug in npm (or at least npm on your machine).

[JSMike]

the node_modules/esbuild/bin/esbuild file was corrupted. If you're experiencing this issue then compare the bin/esbuild file from the public npm with your company's npm and see if you can get library re-uploaded. In my case the bin/esbuild file for our on-prem solution somehow became an actual binary that is 9mb.

This is a deliberate optimization and is how the installer normally works (if --ignore-scripts is not present, if you aren't using Yarn, and if you're not on Windows). The source code for this is here if you're curious (including a comment about when and why it happens). That does not indicate that the package has been "corrupted". As you noted, it should not happen with --ignore-scripts because of course the install script should not be run. If the install script is still being run with --ignore-scripts then there's some bug in npm (or at least npm on your machine).

The issue must be with our on-prem npm registry then, and/or our process for automating the syncing of libraries to the on-prem registry. I can only assume that the script my company is using to sync from global npm is doing an npm install on a linux box and then uploading the result to the registry. On windows I'm seeing the linux binary file even when using --ignore-scripts when it should match https://unpkg.com/[email protected]/bin/esbuild

[JSMike]

FYI, confirmed the tarball in my company's npm-registry has the linux binary at /bin/esbuild and not the script.

I know you mentioned that the result was as designed after the post-install step, but is there a pattern you could follow that doesn't overwrite the file?

I'll be working with my internal ops team to resolve the issue for us, but I'm sure this will impact more than just my company.

[JSMike]

That does not indicate that the package has been "corrupted". As you noted, it should not happen with --ignore-scripts

And yes, sorry, I didn't mean that your package was corrupted, but that the tarball in my company's npm registry was invalid.

[evanw]

is there a pattern you could follow that doesn't overwrite the file?

Isn't --ignore-scripts that pattern? Whoever wrote the code for the registry processor could install packages with the --ignore-scripts flag to avoid incorrectly baking in platform-specific details before they snapshot the package contents on the file system.

[JSMike]

I'll be working with the team that manages the registry at our company to fix their script. What we're doing is non-standard.

I still think that maybe you should consider not overwriting the file, as we may not be the only ones impacted (say if someone does something silly and zips their project over to another machine after post-install, or the filesystem gets mounted on a different host with something like docker). There should be a way to npm rebuild and get the correct binary. The current pattern fails npm rebuild if the binary has been overwritten.

Either way, thank you for making an awesome library! 😄