Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v2.8.28's .tgz contains corrupt timestamps, causing errors from zip command and others #2065

Closed
jesstelford opened this issue Jun 7, 2017 · 12 comments

Comments

@jesstelford
Copy link

jesstelford commented Jun 7, 2017

Bug report or feature request?

Bug report

Uglify version (uglifyjs -V)

$ ./node_modules/.bin/uglifyjs -V
uglify-js 2.8.28

Steps to reproduce

$ npm init -y
$ npm install [email protected]
$ ls node_modules/uglify-js
total 104
-rw-r--r--   1 jess.telford  staff   1.3K  1 Jan  1970 LICENSE
-rw-r--r--   1 jess.telford  staff    41K  1 Jan  1970 README.md
...

Notice the Jan 1st, 1970 timestamp.

This is causing issues on certain platforms where the timestamp is actually reported as December 31st, 1969, causing POSIX tools to fail. (ie; it's a -1 unix timestamp).

Edit: The actual error is: ZIP does not support timestamps before 1980 on Ubuntu 14.04 (our build machine)

Analysis

It looks like it's the actual files inside the package, which implies it's not anything to do with npm, but instead to do with the state of the filesystem at the time of publish Edit: It was npm! (I've accidentally caused bad publishes like this myself in the past):

$ curl https://registry.npmjs.org/uglify-js/-/uglify-js-2.8.28.tgz | tar -tvf -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0-rw-rw-rw-  0 0      0        1139  1 Jan  1970 package/package.json
-rw-rw-rw-  0 0      0       41915  1 Jan  1970 package/README.md
-rw-rw-rw-  0 0      0        1348  1 Jan  1970 package/LICENSE
-rw-rw-rw-  0 0      0        1982  1 Jan  1970 package/bin/extract-props.js
-rw-rw-rw-  0 0      0       21426  1 Jan  1970 package/bin/uglifyjs
-rw-rw-rw-  0 0      0       11096  1 Jan  1970 package/lib/utils.js
-rw-rw-rw-  0 0      0        6938  1 Jan  1970 package/lib/transform.js
-rw-rw-rw-  0 0      0        3644  1 Jan  1970 package/lib/sourcemap.js
-rw-rw-rw-  0 0      0       23483  1 Jan  1970 package/lib/scope.js
-rw-rw-rw-  0 0      0        8895  1 Jan  1970 package/lib/propmangle.js
-rw-rw-rw-  0 0      0       57351  1 Jan  1970 package/lib/parse.js
-rw-rw-rw-  0 0      0       47999  1 Jan  1970 package/lib/output.js
-rw-rw-rw-  0 0      0       22265  1 Jan  1970 package/lib/mozilla-ast.js
-rw-rw-rw-  0 0      0      179487  1 Jan  1970 package/lib/compress.js
-rw-rw-rw-  0 0      0       34888  1 Jan  1970 package/lib/ast.js
-rw-rw-rw-  0 0      0       10194  1 Jan  1970 package/tools/node.js
-rw-rw-rw-  0 0      0         688  1 Jan  1970 package/tools/exports.js
100  126k  100  126k    0     0   232k      0 --:--:-- --:--:-- --:--:--  232k

-rw-rw-rw-  0 0      0        1640  1 Jan  1970 package/tools/props.html

Note that this is not a problem in 2.8.27:

$ curl https://registry.npmjs.org/uglify-js/-/uglify-js-2.8.27.tgz | tar -tvf -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0-rw-rw-rw-  0 0      0        1139 19 May 19:53 package/package.json
-rw-rw-rw-  0 0      0       41915 19 May 19:53 package/README.md
-rw-rw-rw-  0 0      0        1348 15 Jan 21:32 package/LICENSE
-rw-rw-rw-  0 0      0        1982 19 May 19:53 package/bin/extract-props.js
-rw-rw-rw-  0 0      0       21426 19 May 19:53 package/bin/uglifyjs
-rw-rw-rw-  0 0      0       34888 19 May 19:53 package/lib/ast.js
-rw-rw-rw-  0 0      0      179487 19 May 19:53 package/lib/compress.js
-rw-rw-rw-  0 0      0       22265 19 May 19:53 package/lib/mozilla-ast.js
-rw-rw-rw-  0 0      0       47845 19 May 19:53 package/lib/output.js
-rw-rw-rw-  0 0      0       57351 19 May 19:53 package/lib/parse.js
-rw-rw-rw-  0 0      0        8895 19 May 19:53 package/lib/propmangle.js
-rw-rw-rw-  0 0      0       23483 19 May 19:53 package/lib/scope.js
-rw-rw-rw-  0 0      0        3644 22 Mar 22:18 package/lib/sourcemap.js
100  127k  100  127k    0     0   132k      0 --:--:-- --:--:-- --:--:--  132k

-rw-rw-rw-  0 0      0       11096 19 May 19:53 package/lib/utils.js
-rw-rw-rw-  0 0      0         688 19 May 19:53 package/tools/exports.js
-rw-rw-rw-  0 0      0       10194 19 May 19:53 package/tools/node.js
-rw-rw-rw-  0 0      0      142838 19 May 19:53 package/tools/domprops.json
-rw-rw-rw-  0 0      0        1640 15 Jan 21:32 package/tools/props.html

Edit: Related to #2054

Edit 2: here's an incomplete list of other issues that appear to have a root cause as outlined above:

@alexlamsl
Copy link
Collaborator

Closing as duplicate of #2054

(I will say this one more time - if somebody can figure out why npm@5 would create tar ball containing files of what looks like UTC date of 0 that would help. Otherwise this is bound to happen again in the next npm publish.)

@fdintino
Copy link

fdintino commented Jun 7, 2017

@alexlamsl have you not read any of the discussion? This doesn't have anything to do with npm@5! npm publish doesn't change the file metadata of the local files, it uses them as it finds them in the directory where it is run. The reason it created a tar file with files created and modified at epoch 0 is because they were that way on the filesystem where it was run. I can only speculate about how they got that way on your machine.

I guarantee that if you clone the repository fresh into a different directory, bump the version, build, and publish, the files will not have UTC date of 0. REGARDLESS of which npm version you use.

@jesstelford
Copy link
Author

Note: I have moved this reproducible case to the other issue which appears to be getting more attention.

@gabor
Copy link

gabor commented Jun 13, 2017

maybe npm/npm#16734 ?

@alexlamsl
Copy link
Collaborator

maybe npm/npm#16734 ?

@gabor I think you've just hit jackpot - thanks for finding the root cause of this issue 👍

@kzc
Copy link
Contributor

kzc commented Jun 13, 2017

maybe npm/npm#16734 ?

An npm maintainer commented that it's a node 8 bug that was fixed but it's not clear what node 8 bug that is, or what release it will be in. Probably best to avoid node@8 and npm@5 for releases until this shakes out.

@not-an-aardvark
Copy link

It was apparently fixed by nodejs/node#13173, which was released as part of Node 8.1.0.

@fdintino
Copy link

fdintino commented Jun 13, 2017

Nice find @gabor! It looks like I was very much in the wrong about this not being a node bug.

@alexlamsl I hope this means there will be a re-release of 2.x packaged either with node 8.1 or node 6.x. This is not for my benefit (we resolved this issue in our code days ago, shortly after I posted in this thread, by pinning to @2.8.27 in our package.json) but for the benefit of others who have not yet been bitten by the ctime and mtime in the 2.8.28 release's tar file—that they don't have to spend time troubleshooting and diagnosing why zip or unison isn't working with a build of their project.

@jesstelford
Copy link
Author

Excellent debugging @gabor.

@alexlamsl are you able to confirm it was that particular combo of node@>8.0.0<8.10 and Windows 7 which triggered the bug causing the 0 timestamp?

If so, can we assume a new release with a node version outside of that range would fix it?

@alexlamsl
Copy link
Collaborator

@jesstelford not quite Win7, but I was certainly using node/8.0.0/x64 on Windows.

So in the next backports release, I'll be using node@6 (LTS) for npm publish instead.

@STRML
Copy link
Contributor

STRML commented Jun 13, 2017

Since this release is clearly broken for some users (the last thread makes that obvious), and the root cause has been found, would you please publish2.8.29 as a repub of 2.8.28 using node@6? Republishing the same code under a new tag to fix a publish error is common (I've done it once or twice) and that will calm everyone until the next release.

@alexlamsl
Copy link
Collaborator

@STRML sounds like a plan - let me get out of bed first 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants