npm install of 2.8.28 is downloading files with timestamp of Dec 31, 1969

[et304383]

[etucker: npm_test]$ ll
total 0
[etucker: npm_test]$ npm install [email protected]
npm WARN saveError ENOENT: no such file or directory, open '/home/etucker/play/npm_test/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/home/etucker/play/npm_test/package.json'
npm WARN npm_test No description
npm WARN npm_test No repository field.
npm WARN npm_test No README data
npm WARN npm_test No license field.

+ [email protected]
added 17 packages in 2.86s
[etucker: npm_test]$ ll node_modules/uglify-js/
total 64
drwxrwxr-x 2 etucker etucker  4096 Jun  5 11:57 bin
drwxrwxr-x 2 etucker etucker  4096 Jun  5 11:57 lib
-rw-rw-r-- 1 etucker etucker  1348 Dec 31  1969 LICENSE
-rw-rw-r-- 1 etucker etucker  2086 Jun  5 11:57 package.json
-rw-rw-r-- 1 etucker etucker 41915 Dec 31  1969 README.md
drwxrwxr-x 2 etucker etucker  4096 Jun  5 11:57 tools

This does not happen with 3.x:

[etucker: npm_test]$ rm -rf *
[etucker: npm_test]$ npm install uglify-js
npm WARN saveError ENOENT: no such file or directory, open '/home/etucker/play/npm_test/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/home/etucker/play/npm_test/package.json'
npm WARN npm_test No description
npm WARN npm_test No repository field.
npm WARN npm_test No README data
npm WARN npm_test No license field.

+ [email protected]
added 4 packages in 1.417s
[etucker: npm_test]$ ll node_modules/uglify-js/
total 60
drwxrwxr-x 2 etucker etucker  4096 Jun  5 11:57 bin
drwxrwxr-x 2 etucker etucker  4096 Jun  5 11:57 lib
-rw-rw-r-- 1 etucker etucker  1348 Jan 15 06:32 LICENSE
-rw-rw-r-- 1 etucker etucker  1846 Jun  5 11:57 package.json
-rw-rw-r-- 1 etucker etucker 39419 Jun  1 08:33 README.md
drwxrwxr-x 2 etucker etucker  4096 Jun  5 11:57 tools
[etucker: npm_test]$ 

[alexlamsl]

I'm afraid not much can be done over here - may be try asking npm about this?

https://github.com/npm/npm/issues

[alexlamsl]

One thing I can add is uglify-js 3.0.15 was published with npm that was bundled with node 7.10.0, whereas IIRC uglify-js 2.8.28 was done with node 8.0.0 - so you may add that to your issue report for them.

[et304383]

Pretty sure the issue is on your side. 2.8.27 is fine as well. It's only 2.8.28 that is "corrupted."

[alexlamsl]

Unless the package is unusable, I wouldn't be too bothered by it. And there is no way I can influence the behaviour of npm publish, so your best course of action is to ask them about this.

[et304383]

The package is not usable. I cannot zip up the build folder because zip cannot handle files with a creation date before 1980.

I had to switch to 2.8.27. Can you not just republish 2.8.28 to fix these timestamps?

[alexlamsl]

npm does not allow re-publishing the same version. Why does using uglify-js involves creating a zip file anyway? I'm under the impression that it's npm install uglify-js followed by require("uglify-js") in your code somewhere?

[et304383]

So that I can create a package for deploying the application along with npm dependencies to a test environment.

Edit: can you try pushing a new version, 2.8.29?

[alexlamsl]

And even if I were to publish 2.8.29 which is identical to 2.8.28, what guarantees npm would produce the correct result this time?

As advised above, please contact npm to make sure the issue is diagnosed/resolved first.

[et304383]

I can almost guarantee that if I open a ticket with npm and/or nodejs about a specific npm module on a specific version having corrupted timestamps they're going to point the finger back at this module.

[et304383]

I just downloaded your tgz file DIRECTLY from npm, and it's corrupted IN the tgz file:

[etucker: npm_test]$ npm view [email protected] | tail -n 10
  optionalDependencies: { 'uglify-to-browserify': '~1.0.0' },
  browserify: { transform: [ 'uglify-to-browserify' ] },
  scripts: { test: 'node test/run-tests.js' },
  gitHead: '23876a84a51835ca791afa12931e747df048178f',
  dist: 
   { integrity: 'sha512-WqKNbmNJKzIdIEQu/U2ytgGBbhCy2PVks94GoetczOAJ/zCgVu2CuO7gguI5KPFGPtUtI1dmPQl6h0D4cPzypA==',
     shasum: 'e335032df9bb20dcb918f164589d5af47f38834a',
     tarball: 'https://registry.npmjs.org/uglify-js/-/uglify-js-2.8.28.tgz' },
  directories: {} }

[etucker: npm_test]$ wget https://registry.npmjs.org/uglify-js/-/uglify-js-2.8.28.tgz
--2017-06-05 13:59:16--  https://registry.npmjs.org/uglify-js/-/uglify-js-2.8.28.tgz
Resolving registry.npmjs.org (registry.npmjs.org)... 151.101.0.162, 151.101.64.162, 151.101.128.162, ...
Connecting to registry.npmjs.org (registry.npmjs.org)|151.101.0.162|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 129468 (126K) [application/octet-stream]
Saving to: ‘uglify-js-2.8.28.tgz’

uglify-js-2.8.28.tgz                      100%[==================================================================================>] 126.43K  --.-KB/s    in 0.07s   

2017-06-05 13:59:16 (1.69 MB/s) - ‘uglify-js-2.8.28.tgz’ saved [129468/129468]

[etucker: npm_test]$ tar xf uglify-js-2.8.28.tgz 
[etucker: npm_test]$ ll
total 132
drwxrwxr-x 5 etucker etucker   4096 Jun  5 13:59 package
-rw-rw-r-- 1 etucker etucker 129468 Jun  3 10:54 uglify-js-2.8.28.tgz
[etucker: npm_test]$ ll package
total 64
drwxrwxr-x 2 etucker etucker  4096 Jun  5 13:59 bin
drwxrwxr-x 2 etucker etucker  4096 Jun  5 13:59 lib
-rw-rw-r-- 1 etucker etucker  1348 Dec 31  1969 LICENSE
-rw-rw-r-- 1 etucker etucker  1139 Dec 31  1969 package.json
-rw-rw-r-- 1 etucker etucker 41915 Dec 31  1969 README.md
drwxrwxr-x 2 etucker etucker  4096 Jun  5 13:59 tools
[etucker: npm_test]$ 

[et304383]

Regardless of who is to blame here, you have to resolve this somehow. You have a corrupted tarball on the latest 2.x version available. It's not my responsibility to figure out why your publish corrupted the timestamps on these files.

[fdintino]

Why is this issue closed? The package you published to npm has corrupt file metadata. There are developers here at The Atlantic who have lost a whole afternoon of work trying to debug an issue caused by 2.8.28.

[fdintino]

@alexlamsl npm publish will package up whatever files you have locally. That would have been where the problem was introduced, in the directory on your filesystem where you did the npm publish. If you were to publish 2.8.29 from a fresh clone of the repository I'm sure there would be no issue with the files.

[kzc]

There are developers here at The Atlantic who have lost a whole afternoon of work trying to debug an issue caused by 2.8.28.

What issue?

$ node_modules/.bin/uglifyjs -V
uglify-js 2.8.28

$ echo 'console.log(1 + 2);' | node_modules/.bin/uglifyjs -c
console.log(3);

2.8.28 still functions correctly regardless of the bad timestamps produced by npm publish with node 8/npm 5.

[fdintino]

It caused errors with the filesystem in our dockerized development environment. Of course it still functions as it's supposed to, provided the package can actually be installed.

[kzc]

It caused errors with the filesystem in our dockerized development environment. Of course it still functions as it's supposed to, provided the package can actually be installed.

The next 2.x release can be made with a non-npm@5 version of npm, but the issue you're describing concerns the docker project or is a bug with npm on the docker filesystem.

[daemon1981]

Same issue here. Our private deployment system reject the tarball.

If I extract the tarball and stat package/README.md for instance, I get an incorrect/strange date :

16777220 33434588 -rw-r--r-- 1 francetv wheel 0 41915 "Jun  6 16:03:41 2017" "Jan  1 01:00:00 1970" "Jun  6 16:03:41 2017" "Jan  1 01:00:00 1970" 4096 88 0 package/README.md

[et304383]

@alexlamsl please stop trying to defer blame here. The issue is with this specific npm package for a specific version. Push a new version so people can move on.

[daemon1981]

tar tvf uglify-js-2.8.28.tar gives :

-rw-rw-rw-  0 0      0        1139  1 jan  1970 package/package.json
-rw-rw-rw-  0 0      0       41915  1 jan  1970 package/README.md
-rw-rw-rw-  0 0      0        1348  1 jan  1970 package/LICENSE
-rw-rw-rw-  0 0      0        1982  1 jan  1970 package/bin/extract-props.js
-rw-rw-rw-  0 0      0       21426  1 jan  1970 package/bin/uglifyjs
-rw-rw-rw-  0 0      0       11096  1 jan  1970 package/lib/utils.js
...

Should I post a tar issue ?

[mishagray]

This is blocking my deployments also...

Any workaround?

[et304383]

@mishagray use 2.8.27 until the module author decides that this isn't a problem with node / npm.

[rvanvelzen]

This seems related to npm/npm#10052, which (if resolved) would alleviate your issue.

[fdintino]

It's true that npm/npm#10052 would incidentally fix this issue, because it would reset modification times on files, but there is a much simpler fix that would not require waiting for an 18-month-old issue on npm to be fixed. Make a fresh clone of the repository, bump the version, and publish to npm. I am certain that whatever condition caused the initially packaged files to have corrupted metadata will not be reproduced and the problem will be solved. It doesn't matter what version of npm you use to publish because it is not a bug in npm.

This is a bug in the package you pushed to the npm registry, @alexlamsl, plain and simple. Your refusal to acknowledge this or to take even the simplest steps to address it reflects very poorly on you and on this project.

[rvanvelzen]

@fdintino This is not a bug in UglifyJS.

[et304383]

Who cares what the cause of the issue is? Fix it first then point fingers after. This attitude of "not my problem" when your package is broken solves nothing.

[rvanvelzen]

I'm baffled by your attitude, frankly. An immeasurable amount of time has been put into this project by people in their own spare time, and you expect them to fix your environment for you?

UglifyJS is not broken.

(And to be honest, it is exactly this attitude that drives me away from doing OSS at all)

[et304383]

Fix MY Environment? Wtf are you talking about? It's been demonstrated that the tarball on npm has corrupted timestamp data. How is that MY environment?

[fdintino]

I'm baffled by your attitude, frankly.

Then the bafflement is mutual. You have four different people telling you that the file metadata in the 2.8.28 package is causing issues with their environments and deployments. And you have a (likely) easy solution—push a bumped version from a directory other than the corrupted one that published 2.8.28. But you ignore the former and don't seem to care about the latter. It goes without saying that everyone here appreciates the hard work that is poured into this project, otherwise we wouldn't be using it! I maintain several popular open source projects as well, and I appreciate the challenges that come with it. But if a package I published caused issues in multiple peoples environments, and it was in my power to fix it, I would do so!

I think it's safe to assume that the odd modification time of the files in uglify-js is not intentional—it has no functional benefit and it's also plainly incorrect, these files were not modified in 1969. You have multiple people telling you that this causes incompatibilities in various different systems.

I can only speak for myself, but here's a demonstration of how the files are incompatible in our system. To reproduce, you just need to install unison, a tool for performing two-way directory syncing:

mkdir /tmp/{a,b}
cd /tmp/a
npm init -y
npm install --save [email protected]
cd ..
unison a/ b/ -auto -batch -times -prefer newer || echo "EXITED WITH ERROR CODE $?"

Outputs the following:

Contacting server...
Looking for changes
Reconciling changes
new dir  ---->            node_modules
new file ---->            package.json
Propagating updates

UNISON 2.48.4 started propagating changes at 17:48:06.09 on 06 Jun 2017

[BGN] Copying node_modules from /tmp/test/a to /tmp/test/b

Failed: Failed to set modification time of file
/tmp/test/b/.unison.node_modules.08b6882f.unison.tmp/uglify-js/bin/extract-props.js
to 1969-12-31 at 19:00:00 (0.000000): the time was set to 2017-06-06 at 17:48:06
(1496785686.000000) instead

99%  00:00 ETA

Failed [node_modules]: Failed to set modification time of file
/tmp/test/b/.unison.node_modules.08b6882f.unison.tmp/uglify-js/bin/extract-props.js
to 1969-12-31 at 19:00:00 (0.000000): the time was set to 2017-06-06 at 17:48:06
(1496785686.000000) instead

[BGN] Copying package.json from /tmp/test/a to /tmp/test/b
[END] Copying package.json
UNISON 2.48.4 finished propagating changes at 17:48:06.12 on 06 Jun 2017
Saving synchronizer state
Synchronization incomplete at 17:48:06  (1 item transferred, 0 skipped, 1 failed)
  failed: node_modules
EXITED WITH ERROR CODE 2

Is it a bug that unison cannot sync files with a ctime and mtime of 0? Yes, perhaps, and I can open a ticket on that project. But judging by the others who have posted on this issue, the file metadata in the npm tgz are causing problems in other systems. And there is an easy way to ensure compatibility with everyone's system—a clean build and publish of a new version. I apologize for my tone earlier in this thread—I'm just frustrated by the refusal to even try to help.

[kzc]

Fix MY Environment? Wtf are you talking about? It's been demonstrated that the tarball on npm has corrupted timestamp data. How is that MY environment?

@eric-tucker: "I cannot zip up the build folder because zip cannot handle files with a creation date before 1980."

Even with the incorrect timestamps, npm install [email protected] works on every platform I've tried and runs correctly.

The problem with your zip program is out of scope of uglify. It's trivial to write a command to touch the files for your broken zip command.

[et304383]

I will offer my apology regarding assuming this wasn't an npm issue and for any hostility shown previously.

However, I stand by my affirmation that this should have been handled more professionally (by everyone), including keeping the issue open while people tracked down the root cause.

Edit: side note - I wouldn't have been able to create any kind of helpful issue on npm as I have no idea what the author's build environment and/or npm/node versions were/are at the time of publish. The author should have created the ticket with npm saying "this is my environment and this is what happened."

[joepie91]

My two cents here:

It's important for maintainers of open-source projects to accept bugs as valid, even if they are not directly caused by their own code. Given that the maintainers are the only people who are intimately familiar with the codebase and all of its dependencies, they are also the only people who are in a good position to report bugs upstream and track them, essentially acting as a middleman towards their users.

This is also why issues should only be closed if there's clear evidence that they're invalid, not just "looks like not our problem". If there's doubt about whether it's user error or a bug in the maintainer's stack, then it's the job of the maintainer to ask the right questions to assess that (and prepare issue templates, and so on). The user can't know what information you need, until you ask.

Put more succinctly: the user doesn't care whether it's the fault of the maintainer or not. They also shouldn't need to care, for open-source software to be sustainable and competitive in the long run.

That having been said, there's a big difference between informing a maintainer that it's their job to handle bug reports, and personally accusing them of being responsible for your woes. Especially when you're a developer (read: a professional), you're expected to have the infrastructure in place to deal with upstream issues when they inevitably occur.

While it is the maintainer's responsibility to pick up the bug and trace where it originates from, they are not responsible for your loss of work time, unless you have a support contract with an SLA.

In the end, nothing is accomplished by pointing blame back and forth. Let's just each take our part of the responsibility - maintainers are responsible for tracking down bugs and getting them fixed regardless of where in the stack they occur, and developers ("users") are responsible for having their own contingency plans when things do eventually break, despite best efforts of all parties.

[fdintino]

@joepie91 all excellent points. I would add though that, while I can only speak for myself, my expectation isn't that the uglifyjs maintainers should fix this for me. Yes it cost me some time to troubleshoot and then fix myself, but I committed a fix to our codebase shortly after my first comment in this issue thread. I agree with you that this is my responsibility. What concerns me are the countless people who have run into this issue but haven't spoken up, or who are yet to encounter it. Particularly now that the problem has been identified and the solution is clear (push out a bumped version with either node 6.x or 8.1.0), it still bothers me that there is no will to fix this before the next backport release.

[et304383]

Very good points @joepie91 .

I would agree with @fdintino that waiting until the next backport release to address this problem is definitely downplaying the severity of this issue. It's almost always brought in as a transitive dependency so for someone starting a new project who hasn't yet encountered this issue they are most certainly going to encounter this bug. This will continue to consume a lot of development hours across many companies until 2.8.29+ is released.

I agree with using open source is always "at your risk" but this is an incredibly important issue that still remains closed when it's been confirmed to be an issue with the author's publish environment (even if not caused by the author).

Closed means non-issue, can't reproduce, or some other status of "not something we can or will address." This issue falls under none of those categories.

[pythoneer]

Just to recap and to get a more clear vision of how we can proceed. The issue will not be fixed, although the problem is identified. At least until the next backport release? Is there any vague time frame when this happens? I am asking because we have no easy way to fix the current corrupted package by ourself in our current build chain. Any of the current workarounds flying around here are not really applicable in the way our build process works currently. So the only way to get our system running again is removing dependencies that pulled in UglifyJS2 and try to write the functionality ourself? I would be glad to get a statement about this so we can make an informed decision and work out a solution plan and start taking action to get our system back to life.

[fdintino]

@pythoneer if you npm install --save [email protected] and you are using npm >= 3.x then it will satisfy all intermediate dependencies that have 'uglify-js': '~2.8.x' or 'uglify-js': '^2.8.x' in their package.json. And as a proactive measure you can use npm shrinkwrap or yarn to recursively freeze all dependencies, to guard against future backwards-breaking changes in other packages.

[ZigMeowNyan]

@alexlamsl

When there's a complaint about a specific package version like this, please consider either using one of the many npm API tools to directly download the package and verify it, or using npm pack to verify the packaging behavior yourself. Either would have let you confirm the issue relatively quickly and could have prevented a lot of this chatter. Closing it and leaving it closed was not appropriate, as the issue still exists in a published package version that you've left as the latest in that version range. Even if the root of the problem is upstream, that doesn't resolve the current problem users have with the published version.

One thing I can add is uglify-js 3.0.15 was published with npm that was bundled with node 7.10.0, whereas IIRC uglify-js 2.8.28 was done with node 8.0.0 - so you may add that to your issue report for them.

I understand that the root cause of this issue wasn't in uglify-js. But the tools you use when you build packages is still your responsibility. If you're using a tool (or a version of that tool) that produces incorrect output, the expectation is that you recall the release and/or you downgrade your tools until that issue is fixed - not that you leave everyone in limbo until upstream figures out what your problem is and fixes it for you. If you're going to wait on upstream, unpublishing or retagging the offending build should be a strong suggestion. Just letting people sit with broken builds isn't your only option. And it shouldn't be an option at all. Especially on a maintenance branch like this.

Many developers (myself included) like to use the latest versions of their tools and software, and that's understandable. But sometimes this can introduce bugs. I'd like to suggest that you set up a free account with https://travis-ci.org/. You can do automated builds on there and set it up to publish directly from there, so the setup on your local machine won't matter at that point. If your personal setup is going to change frequently, it makes sense to use a more static configuration that's not as fragile.

Good luck in dealing with the remaining pieces of the issue.

[jrgp]

Obligatory XKCD:

[et304383]

Is it a matter of pride that the issue is still closed? I've already implemented fixes in our build processes and told the company wide to lock any new package.json entries to version 2.8.27. That doesn't mean that a new point release shouldn't be published immediately.

Yes, it is a small inconvenience in terms of individual persons or projects, but you should consider the collective time spent across all people dealing with this issue, including the numerous other projects having reported issues that lead back to this issue (that leads back to the fixed npm issue).

@alexlamsl please, at least reopen this issue, and mark it fixed when you have time to update node/npm and republish. This is going to continue to break builds until a fix is implemented. Put at least a little consideration into the time of all the people who will ultimately see this issue pop up.

[djt-code]

@eric-tucker I'm glad that you were able to fix your build process, but I don't think it's a productive use of your time to continue repeating your opinions here. I'm pretty certain you've made your beliefs crystal clear. I think that everyone on both sides of this have had ample opportunity to express their opinions, and many valid points were made from both sides. At this point, the only logical thing to do is to sit back and see what the maintainers do. I do not believe that continuing to berate them will offer much incentive for them to grant your wishes.

And always remember people:

[StoneCypher]

If people had put as much effort into being helpful as into writing meme comments, this would have been fixed a week ago.

I never actually had this problem, but the response of the maintainers and community is making me think that I am unsafe to depend on this tool.

[UglifyJS-Flame-is-a-Meme]

[schwitzerm]

Time to move to Clojure Compiler, this debacle is embarrassing.

[fdintino]

@schwitzerm A fixed release was pushed out to npm, 2.8.29. I really appreciate it @alexlamsl. I don't benefit from it personally, but I'm glad this won't cause issues going forward for fellow developers. You have gotten a lot of grief (some, admittedly, from me). And you aren't likely to be thanked by any of the people who would have been frustrated by the node bug that crept into the 2.8.28 build but now will not, because people only notice when things break. The same goes for all of the other bug fixes you continue to push out for the 2.x backport branch and the features being added to 3.x. I would imagine, particularly after this episode, that this can sometimes be a thankless job. So I'd like to personally express my gratitude for your work.

[daemon1981]

[divs1210]

@alexlamsl this is honestly the most brutal silent treatment from a maintainer of a popular repo I've seen till now.

[pink-mist]

With the amount of pisspoor attitude from most people in this thread, can you blame them? Also, this was never a problem caused by this project, it was always third party tools that couldn't handle timestamps properly, and the only reason it was found out was because of a bug in yet another tool. None of this was caused because of UglifyJS.

However, they already have released a fixed version. https://github.com/mishoo/UglifyJS2/releases/tag/v2.8.29

So why are you still harping on on this obsolete and outdated issue that doesn't mean anything to anyone anymore?