win,build: add creation of zip and 7z package #5995

joaocgreis · 2016-04-01T12:02:15Z

Pull Request check-list

Does make -j8 test (UNIX) or vcbuild test nosign (Windows) pass with
this change (including linting)?
Is the commit message formatted according to [CONTRIBUTING.md][0]?

Affected core subsystem(s)

Windows, build.

Description of change

Add a step in vcbuild.bat to create a minimal package including node and npm that can be used as an alternative to the MSI, compress the node.pdb file as zip and 7z and upload all files as part of build-release.

This requires 7zip to be installed on the release machines. If there is no objection, I plan to add it and test this with a nightly. Files are included in the upload stage, do we need to do anything else to add them to releases @nodejs/build @nodejs/release ?

Fixes: nodejs/build#299
Fixes: #5696

cc @nodejs/platform-windows

richardlau · 2016-04-01T13:31:34Z

Should the zip also contain Node's license file?

bzoz · 2016-04-05T17:41:40Z

Updated missing files, PTAL

joaocgreis · 2016-04-11T13:57:48Z

@nodejs/build added 7zip to the release machines.

Here is a test build: https://nodejs.org/download/test/v6.0.0-test20160411758655d70e88446a24d8dbbb07f1c1defd317293/

rvagg · 2016-04-11T15:07:18Z

I think the bundle files should go in the parent directory, leaving win-x* for individual files (including the zipped pdb)

benjamingr · 2016-04-11T20:56:56Z

Silly question perhaps, but are we completely OK here in terms of licensing? I remember 7z is an open LGPL format and IIRC there are open clients for opening it on all supported platforms but I want to make sure someone "from the loop" is aware.

bzoz · 2016-04-12T07:59:47Z

Moved the archives to parent directory.

joaocgreis · 2016-04-12T14:40:11Z

@benjamingr I'm not completely sure about anything related to licences. We are only using 7 Zip the executable, not linking against it or the dll. Is it an issue?

benjamingr · 2016-04-12T15:10:49Z

I'm 99% sure that this is not an issue and I'm probably a lot more clueless than you about it - but since I haven't seen 7z files in the project before I think it would be a good idea to make sure with someone who is sure.

orangemocha · 2016-04-12T15:17:32Z

@mikeal can you maybe help answer the licensing concern?

joaocgreis · 2016-05-05T11:22:23Z

From http://www.7-zip.org/license.txt :

Note:
You can use 7-Zip on any computer, including a computer in a commercial
organization. You don't need to register or pay for 7-Zip.

Hence, we can move ahead if there are no more specific concerns.

joaocgreis · 2016-05-05T11:57:25Z

Rebased, a new test build is building: https://nodejs.org/download/test/v7.0.0-test20160505ef2e444e6dc4401519f2f28bcaf6d6b48b8f3590/

joaocgreis · 2016-05-05T12:03:59Z

@nodejs/build @nodejs/platform-windows this is still in need of a LGTM, can you take a look?

joaocgreis · 2016-05-05T22:14:36Z

@rvagg After moving the zip and 7z files out of the win-x* directory, they are left in the staging server and do not appear online. Can you point us to the correct place to change to make this work? Thanks!

Add a step in vcbuild.bat to create a minimal package including node and npm that can be used as an alternative to the MSI. Compress the node.pdb file as zip and 7z. All files are uploaded as part of build-release. Fixes: nodejs/build#299 Fixes: nodejs#5696

rvagg · 2016-05-15T14:00:20Z

@joaocgreis I just ran another test @ https://nodejs.org/download/test/v7.0.0-test2016051536b525a781/ and managed to catch this state in staging before the *.done files were cleaned up—which is done every 5 minutes after the matching files are moved to dist.

-rw-r--r--  1 staging staging  6559346 May 15 09:52 node-v7.0.0-test2016051536b525a781-win-x64.7z
-rw-r--r--  1 staging staging 12007980 May 15 09:52 node-v7.0.0-test2016051536b525a781-win-x64.zip
-rw-rw-r--  1 staging staging        0 May 15 09:52 node-v7.0.0-test2016051536b525a781-x64.7z.done
-rw-rw-r--  1 staging staging        0 May 15 09:52 node-v7.0.0-test2016051536b525a781-x64.zip.done

i.e. it's just a simple mismatch between the .done files and their promotables:

node-v%FULLVERSION%-win-%target_arch%.7z and node-v%FULLVERSION%-win-%target_arch%.zip

but the touch makes:

node-v%FULLVERSION%-%target_arch%.zip.done and node-v%FULLVERSION%-%target_arch%.7z.done

Put the win- in there and it should all work fine and we can run another test.

bzoz · 2016-05-16T13:53:11Z

I've updated the filenames, PTAL

joaocgreis · 2016-05-16T14:45:01Z

@rvagg Thanks!

Started another build, this time it looks good: https://nodejs.org/download/test/v7.0.0-test201605168b733b76c753513ff87ed841e2bef69b12fe0460/

rvagg · 2016-05-17T12:50:18Z

yes, lgtm, but let's give @nodejs/ctc a chance to jump in here before you merge since this is actually not a small deal. Once we start shipping .7z and .zip files for Windows we're not going to be able to undo that very easily in the future, so it's a big commitment.

Our dist directories will start to look like this:

../
docs/
win-x64/
win-x86/
SHASUMS256.txt
node-v7.0.0-darwin-x64.tar.gz
node-v7.0.0-darwin-x64.tar.xz
node-v7.0.0-headers.tar.gz
node-v7.0.0-headers.tar.xz
node-v7.0.0-linux-arm64.tar.gz
node-v7.0.0-linux-arm64.tar.xz
node-v7.0.0-linux-armv6l.tar.gz
node-v7.0.0-linux-armv6l.tar.xz
node-v7.0.0-linux-armv7l.tar.gz
node-v7.0.0-linux-armv7l.tar.xz
node-v7.0.0-linux-ppc64le.tar.gz
node-v7.0.0-linux-ppc64le.tar.xz
node-v7.0.0-linux-ppc64.tar.gz
node-v7.0.0-linux-ppc64.tar.xz
node-v7.0.0-linux-x64.tar.gz
node-v7.0.0-linux-x64.tar.xz
node-v7.0.0-linux-x86.tar.gz
node-v7.0.0-linux-x86.tar.xz
node-v7.0.0.pkg
node-v7.0.0-sunos-x64.tar.gz
node-v7.0.0-sunos-x64.tar.xz
node-v7.0.0-sunos-x86.tar.gz
node-v7.0.0-sunos-x86.tar.xz
node-v7.0.0.tar.gz
node-v7.0.0.tar.xz
node-v7.0.0-win-x64.7z
node-v7.0.0-win-x64.zip
node-v7.0.0-win-x86.7z
node-v7.0.0-win-x86.zip
node-v7.0.0-x64.msi
node-v7.0.0-x86.msi

And both win-x64 and win-x86 have two new files, node_pdb.7z ~ 6.2M and node_pdb.zip ~ 10M.

<= v0.12 we ship(ped) with uncompressed node.pdb files ~31M but we canned that from io.js onward due to skepticism that they were even being used, and we have had minimal call to add them back. Here's the one issue we've had filed about it: #5696. .pdb files contain debugging information for Windows applications.

So, comment now before this all gets locked in.

Fishrock123 · 2016-05-17T14:55:25Z

How large is x86 windows usage? Maybe we could just ship x64 zips? (I have no idea)

Fishrock123 · 2016-05-17T14:56:17Z

How long does it take to produce node_pdb? Will that increase release build time significantly for the windows machines?

joaocgreis · 2016-05-17T15:22:28Z

@Fishrock123 I made a quick experiment, zipping (2 zip + 2 7z) takes 1m21s in our release machines. I don't know exactly how long the upload takes. The test build I ran took 19m, but the Pi1 still takes the longest with 52m.

If the directory size is a concern, we can keep only the MSI in the main dir and move the others into the specific win dirs.

Fishrock123 · 2016-05-17T15:28:03Z

@joaocgreis I'm not worried about the zipping, I was wondering about generating the pdb file.

joaocgreis · 2016-05-17T15:33:29Z

@Fishrock123 the pdb is always generated when compiling, no added delay there.

rvagg · 2016-05-17T20:43:44Z

basic stats are at https://nodejs.org/metrics/summaries/os.png & https://nodejs.org/metrics/summaries/arch.png but there's no combination graph, the data is available @ https://nodejs.org/metrics/ if someone wants to look at the win/arch numbers. My gut feel is that win x86 is still very high.

Re directory size (in terms of number of entries) I'm not so concerned these days, we have a great downloads page that I think most people are using so they don't have to navigate a complex list if they are not capable of doing so.

orangemocha · 2016-05-25T13:46:15Z

LGTM

joaocgreis · 2016-05-30T10:00:52Z

@rvagg ok to land?

rvagg · 2016-05-31T06:55:03Z

lgtm, make it so

Add a step in vcbuild.bat to create a minimal package including node and npm that can be used as an alternative to the MSI. Compress the node.pdb file as zip and 7z. All files are uploaded as part of build-release. Reviewed-By: Joao Reis <[email protected]> Reviewed-By: Rod Vagg <[email protected]> Reviewed-By: Alexis Campailla <[email protected]> PR-URL: #5995 Fixes: nodejs/build#299 Fixes: #5696

joaocgreis · 2016-05-31T11:17:28Z

CI: https://ci.nodejs.org/job/node-test-pull-request/2871/

Landed as 2369f89 .

Add a step in vcbuild.bat to create a minimal package including node and npm that can be used as an alternative to the MSI. Compress the node.pdb file as zip and 7z. All files are uploaded as part of build-release. Reviewed-By: Joao Reis <[email protected]> Reviewed-By: Rod Vagg <[email protected]> Reviewed-By: Alexis Campailla <[email protected]> PR-URL: #5995 Fixes: nodejs/build#299 Fixes: #5696

rvagg · 2016-06-08T06:49:45Z

oh, hey, these all came out in the latest release https://nodejs.org/download/release/latest-v6.x/

Add a step in vcbuild.bat to create a minimal package including node and npm that can be used as an alternative to the MSI. Compress the node.pdb file as zip and 7z. All files are uploaded as part of build-release. Reviewed-By: Joao Reis <[email protected]> Reviewed-By: Rod Vagg <[email protected]> Reviewed-By: Alexis Campailla <[email protected]> PR-URL: nodejs#5995 Fixes: nodejs/build#299 Fixes: nodejs#5696

Add a step in vcbuild.bat to create a minimal package including node and npm that can be used as an alternative to the MSI. Compress the node.pdb file as zip and 7z. All files are uploaded as part of build-release. Reviewed-By: Joao Reis <[email protected]> Reviewed-By: Rod Vagg <[email protected]> Reviewed-By: Alexis Campailla <[email protected]> PR-URL: #5995 Fixes: nodejs/build#299 Fixes: #5696

joaocgreis added windows Issues and PRs related to the Windows platform. build Issues and PRs related to build files or the CI. labels Apr 1, 2016

bzoz force-pushed the Bartosz-MakeWindowsZipPackage branch from 27b681c to 5731fb0 Compare April 5, 2016 17:30

estliberitas force-pushed the master branch 2 times, most recently from 7da4fd4 to c7066fb Compare April 26, 2016 05:23

bzoz force-pushed the Bartosz-MakeWindowsZipPackage branch from 2cdd6fd to ef2e444 Compare May 5, 2016 11:43

bzoz and others added 6 commits May 13, 2016 12:44

add LICENSE, CHANGELOG and README to package

94ea7d4

fixup: replace colon

60e7b00

move archives to parent directories

5a08270

Set proper privileges for uploaded packages

7018770

Add .done files for zip packages

36b525a

joaocgreis force-pushed the Bartosz-MakeWindowsZipPackage branch from ef2e444 to 36b525a Compare May 13, 2016 11:47

Fix .done filenames

8b733b7

gibfahn mentioned this pull request May 18, 2016

win, build: add node.lib file into headers package #6600

Closed

3 tasks

joaocgreis closed this May 31, 2016

joaocgreis added the dont-land-on-v4.x label May 31, 2016

joaocgreis mentioned this pull request May 31, 2016

Backport for v4.x: win,build: add creation of zip and 7z package #7075

Closed

2 tasks

MylesBorins mentioned this pull request Jul 12, 2016

v4.5.0 proposal #7688

Merged

datallah mentioned this pull request Sep 15, 2016

Update to use Windows zip dists when available srs/gradle-node-plugin#142

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

win,build: add creation of zip and 7z package #5995

win,build: add creation of zip and 7z package #5995

joaocgreis commented Apr 1, 2016

richardlau commented Apr 1, 2016

bzoz commented Apr 5, 2016

joaocgreis commented Apr 11, 2016

rvagg commented Apr 11, 2016

benjamingr commented Apr 11, 2016

bzoz commented Apr 12, 2016

joaocgreis commented Apr 12, 2016

benjamingr commented Apr 12, 2016

orangemocha commented Apr 12, 2016

joaocgreis commented May 5, 2016 •

edited

Loading

joaocgreis commented May 5, 2016

joaocgreis commented May 5, 2016

joaocgreis commented May 5, 2016

rvagg commented May 15, 2016

bzoz commented May 16, 2016

joaocgreis commented May 16, 2016

rvagg commented May 17, 2016

Fishrock123 commented May 17, 2016

Fishrock123 commented May 17, 2016

joaocgreis commented May 17, 2016

Fishrock123 commented May 17, 2016

joaocgreis commented May 17, 2016

rvagg commented May 17, 2016

orangemocha commented May 25, 2016

joaocgreis commented May 30, 2016

rvagg commented May 31, 2016

joaocgreis commented May 31, 2016

rvagg commented Jun 8, 2016

win,build: add creation of zip and 7z package #5995

win,build: add creation of zip and 7z package #5995

Conversation

joaocgreis commented Apr 1, 2016

Pull Request check-list

Affected core subsystem(s)

Description of change

richardlau commented Apr 1, 2016

bzoz commented Apr 5, 2016

joaocgreis commented Apr 11, 2016

rvagg commented Apr 11, 2016

benjamingr commented Apr 11, 2016

bzoz commented Apr 12, 2016

joaocgreis commented Apr 12, 2016

benjamingr commented Apr 12, 2016

orangemocha commented Apr 12, 2016

joaocgreis commented May 5, 2016 • edited Loading

joaocgreis commented May 5, 2016

joaocgreis commented May 5, 2016

joaocgreis commented May 5, 2016

rvagg commented May 15, 2016

bzoz commented May 16, 2016

joaocgreis commented May 16, 2016

rvagg commented May 17, 2016

Fishrock123 commented May 17, 2016

Fishrock123 commented May 17, 2016

joaocgreis commented May 17, 2016

Fishrock123 commented May 17, 2016

joaocgreis commented May 17, 2016

rvagg commented May 17, 2016

orangemocha commented May 25, 2016

joaocgreis commented May 30, 2016

rvagg commented May 31, 2016

joaocgreis commented May 31, 2016

rvagg commented Jun 8, 2016

joaocgreis commented May 5, 2016 •

edited

Loading