Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: update Makefile to support fat binary on macos #37861

Closed
wants to merge 3 commits into from

Conversation

AshCripps
Copy link
Member

Opening this PR to track the work it takes to make a fat binary for the new apple arch.

cc @nodejs/build @nodejs/platform-macos

@nodejs-github-bot nodejs-github-bot added build Issues and PRs related to build files or the CI. needs-ci PRs that need a full CI run. labels Mar 22, 2021
@AshCripps AshCripps added the semver-major PRs that contain breaking changes and should be released in the next major version. label Mar 22, 2021
@AshCripps AshCripps force-pushed the apple-uni-binary branch 3 times, most recently from 3090f4c to e63f93e Compare March 23, 2021 17:03
@AshCripps AshCripps marked this pull request as ready for review March 24, 2021 15:33
@AshCripps
Copy link
Member Author

@AshCripps
Copy link
Member Author

https://nodejs.org/download/test/v16.0.0-test20210324b8ae520a83/ heres a sample build I would appreciate people testing @nodejs/platform-macos

@rvagg
Copy link
Member

rvagg commented Mar 25, 2021

I'm not sure this is doing what it should be. The env vars have both host and target being x64, with the only change in here trying to do arm64 being --dest-cpu=arm64 in the second one. I'm pretty sure that convincing GYP to do the right thing should be a lot more complicated than that.
Can you inspect the output of this process and run file dist/x64/node/usr/local/bin/node and file dist/node/usr/local/bin/node to see they are the two different architectures (I'm not sure how you undo lipo but presumably one could inspect the binary from there).
See also nodejs/build#2474 (comment)

@AshCripps
Copy link
Member Author

@rvagg this does produce the two binarys and running file on the lipo binary gives

node/usr/local/bin/node: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64]
node/usr/local/bin/node (for architecture x86_64):	Mach-O 64-bit executable x86_64
node/usr/local/bin/node (for architecture arm64):	Mach-O 64-bit executable arm64

The reason it looks so weird is the first complie runs under rosetta so it thinks everything is intel, the second compile is then run on native arm. I not 100% if its the best solution but it was the first one that worked cleanly without failing for me.

@rvagg
Copy link
Member

rvagg commented Mar 29, 2021

Oh, so you're compiling this on arm64? that would make sense then, neato!
I thought we were blocked on V8 for doing that?

@rvagg
Copy link
Member

rvagg commented Mar 29, 2021

With the weird host & target config for this, it might be best to ensure that this part of Makefile isn't executed unless it's on the specific platform that it needs to be - to avoid any future mixups (or mixups for users who are doing this themselves for whatever reason). So maybe an #error in there for wrong platform?

@AshCripps
Copy link
Member Author

I thought we were blocked on V8 for doing that?

We are unless its done through the rosetta layer so V8 thinks the system is x64

So maybe an #error in there for wrong platform?

will add that in

build a binary that is both x86_64 and arm64 on macos
@rvagg
Copy link
Member

rvagg commented Mar 30, 2021

The other thing I think we need to consider here is that we're now producing x64 binaries on a non-x64 platform but we don't have that in any of our testing paths. How much does it matter? How much should we care? In theory it should be fine, but the fact will be that we're not testing these binaries in any manner in our CI. We've ended up just leaning on cross compiling for armv7 because it took too long to compile natively, but it's also the case that it's very common to consume cross compiled binaries on these platforms.
🤷

@AshCripps
Copy link
Member Author

I did run the test suite against this binary on an intel mac and a m1 mac and all passed - maybe we should set up a citgm-smoker-nobuild style job where we test the latest nightly on both arches to ensure all is well?

Copy link
Member

@mhdawson mhdawson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rvagg
Copy link
Member

rvagg commented Mar 31, 2021

maybe we should set up a citgm-smoker-nobuild style job where we test the latest nightly on both arches to ensure all is well?

better than nothing! it's the nothing I'm concerned about. We currently wouldn't even be able to debug a user's bug reports with CI if anything came up.

There's also the issue of only having one of these M1's available, and in one infra provider, we'd have to be OK with that before proceeding to tie this to the release process for .pkg, it'd lock us in for that.

@AshCripps
Copy link
Member Author

There's also the issue of only having one of these M1's available, and in one infra provider, we'd have to be OK with that before proceeding to tie this to the release process for .pkg, it'd lock us in for that.

Do we not normally only have one release machine available? I thought the 10.15 macs were the exception not the norm.

@mhdawson
Copy link
Member

@AshCripps I think I already asked, but to confirm. The "magic" that makes this work only happens if we run on M1 versus running on x64, right? Just want to understand a bit better why we can't flip it around and build the fat pkg on x86 instead?

@mhdawson
Copy link
Member

@jasnell, @mcollina would it be possible to host a couple of M1's? at NearForm in addition to the ones you currently host. I think I remember that virtualization does not work yet, but if hosting them there is possible I can follow up with the Foundation to see if we can purchase a couple and that would at least give us 1 backup for each of release/test.

@AshCripps
Copy link
Member Author

@mhdawson Yes correct its using the M1 macs and running through the translation layer. I chose the M1 mac because its much faster than any of our x64 machines and I never got very far with testing the cross compilation from x64 to arm.

@rvagg
Copy link
Member

rvagg commented Mar 31, 2021

@mhdawson it should be possible to flip this and do it the other way but the config magic would be more complicated to deal with the host/target distinction. Being able to use Rosetta to solve that problem is brilliant and really simplifies this down quite a bit. These M1s are kind of ideal, not just because of their speed but having Rosetta to help with the cross so I'd +1 leaning in to them after seeing this.

@mcollina
Copy link
Member

mcollina commented Apr 2, 2021

@jasnell, @mcollina would it be possible to host a couple of M1's? at NearForm in addition to the ones you currently host. I think I remember that virtualization does not work yet, but if hosting them there is possible I can follow up with the Foundation to see if we can purchase a couple and that would at least give us 1 backup for each of release/test.

A few of the key people are going on vacation for the Easter, I'll try to get an answer in the next couple of weeks.

@mhdawson
Copy link
Member

mhdawson commented Apr 6, 2021

@rvagg sounds like we should stick to M1, wondering if you need more time to review this PR?

Copy link
Member

@jbergstroem jbergstroem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on a M1, worked for me.

Makefile Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
@nodejs-github-bot

This comment has been minimized.

@mcollina
Copy link
Member

mcollina commented Apr 8, 2021

@jasnell, @mcollina would it be possible to host a couple of M1's? at NearForm in addition to the ones you currently host. I think I remember that virtualization does not work yet, but if hosting them there is possible I can follow up with the Foundation to see if we can purchase a couple and that would at least give us 1 backup for each of release/test.

@mhdawson we can host a couple more mac minis at our HQ. Let's move this offline for shipment etc.

@nodejs-github-bot

This comment has been minimized.

@mhdawson
Copy link
Member

mhdawson commented Apr 8, 2021

@mcollina thanks, I'll have to reach out to the foundation staff to line up/confirm we can fund them and then will co-ordinate with you off line in terms of shipping etc.

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@AshCripps AshCripps added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. and removed needs-ci PRs that need a full CI run. labels Apr 9, 2021
Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

AshCripps pushed a commit that referenced this pull request Apr 9, 2021
build a binary that is both x86_64 and arm64 on macos

PR-URL: #37861
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Beth Griggs <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
@AshCripps
Copy link
Member Author

Landed in 1d8c022

@AshCripps AshCripps closed this Apr 9, 2021
@mhdawson
Copy link
Member

mhdawson commented Apr 9, 2021

just to close out the discussion on M1 machines, 2 M1 Minis have been ordered and are on their way.

@AshCripps AshCripps deleted the apple-uni-binary branch April 15, 2021 10:20
BethGriggs added a commit that referenced this pull request Apr 20, 2021
Notable Changes:

Deprecations and Removals

- **(SEMVER-MAJOR)** **fs**: remove permissive rmdir recursive
  (Antoine du Hamel) [#37216]
- **(SEMVER-MAJOR)** **fs**: runtime deprecate rmdir recursive option
  (Antoine du Hamel) [#37302]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('http_parser') (James M Snell) [#37813]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('url') (James M Snell) [#37799]
- **(SEMVER-MAJOR)** **lib**: make process.binding('util') return only
  type checkers (Anna Henningsen) [#37819]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('crypto') (James M Snell) [#37790]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('signal_wrap') (James M Snell) [#37800]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('v8') (James M Snell) [#37789]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('async_wrap') (James M Snell) [#37576]
- **(SEMVER-MAJOR)** **module**: remove module.createRequireFromPath
  (Antoine du Hamel) [#37201]
- **(SEMVER-MAJOR)** **module**: runtime deprecate subpath folder
  mappings (Antoine du Hamel) [#37215]
- **(SEMVER-MAJOR)** **module**: runtime deprecate "main" index and
  extension lookups (Antoine du Hamel) [#37206]
- **(SEMVER-MAJOR)** **module**: runtime deprecate invalid package.json
  main entries (Antoine du Hamel) [#37204]
- **(SEMVER-MAJOR)** **process**: runtime deprecate changing
  process.config (James M Snell) [#36902]

Stable Timers Promises API:

The Timers Promises API provides an alternative set of timer functions
that return Promise objects. Added in Node.js v15.0.0, in this release
they graduate from experimental status to stable.

Contributed by James Snell - [#38112]

Toolchain and Compiler Upgrades:

Node.js v16.0.0 will be the first release where we ship prebuilt
binaries for Apple Silicon. While we’ll be providing separate tarballs
for the Intel (`darwin-x64`) and ARM (`darwin-arm64`) architectures the
macOS installer (`.pkg`) will be shipped as a ‘fat’ (multi-architecture)
binary.

- **(SEMVER-MAJOR)** **build**: remove support for Python 2
  (Christian Clauss) [#36691]
- **(SEMVER-MAJOR)** **build**: default PYTHON to python3 in Makefile
  (Michaël Zasso) [#37764]
- **build**: update Makefile to support fat binary (Ash Cripps)
  [#37861]
- **(SEMVER-MAJOR)** **build**: enable ASLR (PIE) on OS X (woodfairy)
  [#35704]
- **build**: warn for gcc versions earlier than 8.3.0 (Richard Lau)
  [#37935]
- **(SEMVER-MAJOR)** **doc**: update minimum supported Xcode to 11
  (Michaël Zasso) [#37872]
- **(SEMVER-MAJOR)** **doc**: update minimum supported GCC to 8.3
  (Michaël Zasso) [#37871]
- **(SEMVER-MAJOR)** **doc**: update AIX to GCC8 for v16.x (Ash Cripps)
  [#37677]
- **tools**: set arch in Distribution.xml (Ash Cripps)
  [#38261]

V8 9.0:

The V8 JavaScript engine is updated to V8 9.0, including performance
tweaks and improvements.

This update also brings the ECMAScript RegExp Match Indices, which
provide the start and end indices of the captured string. The indices
array is available via the `.indices` property on match objects when the
regular expression has the `/d` flag.

Contributed by Michaël Zasso - [#37587]

Other Notable Changes:

- **(SEMVER-MINOR)** **assert**: graduate assert.match and
  assert.doesNotMatch (James M Snell) [#38111]
- **(SEMVER-MAJOR)** **buffer**: expose btoa and atob as globals
  (James M Snell) [#37786]
- **deps**: update llhttp to 6.0.0 (Fedor Indutny)
  [#38277]
- **deps**: upgrade npm to 7.10.0 (Ruy Adorno)
  [#38254]
- **(SEMVER-MAJOR)** **deps**: bump minimum ICU version to 68
  (Michaël Zasso) [#37330]
- **(SEMVER-MINOR)** **http**: add http.ClientRequest.getRawHeaderNames()
  (simov) [#37660]
- **(SEMVER-MAJOR)** **lib,src**: update cluster to use Parent
  (Michael Dawson) [#36478]
- **(SEMVER-MINOR)** **module**: add support for `node:`‑prefixed
  `require(…)` calls (ExE Boss) [#37246]
- **(SEMVER-MINOR)** **perf_hooks**: add histogram option to timerify
  (James M Snell) [#37475]
- **(SEMVER-MINOR)** **repl**: add auto‑completion for `node:`‑prefixed
  `require(…)` calls (ExE Boss) [#37246]
- **(SEMVER-MINOR)** **util**: add getSystemErrorMap() impl
  (eladkeyshawn) [#38101]

Semver-Major Commits:

- **async_hooks**: add thisArg to AsyncResource.bind (James M Snell)
  [#36782]
- **buffer**: expose btoa and atob as globals (James M Snell)
  [#37786]
- **build**: remove support for Python 2 (Christian Clauss)
  [#36691]
- **build**: default PYTHON to python3 in Makefile (Michaël Zasso)
  [#37764]
- **build**: update Makefile to support fat binary (Ash Cripps)
  [#37861]
- **build**: reset embedder string to "-node.0" (Michaël Zasso)
  [#37587]
- **build**: include minimal V8 headers in distribution (Michaël Zasso)
  [#37570]
- **build**: reset embedder string to "-node.0" (Michaël Zasso)
  [#37330]
- **build**: reset embedder string to "-node.0" (Michaël Zasso)
  [#36139]
- **build**: use C++11 ABI with libstdc++ (Anna Henningsen)
  [#36634]
- **build**: enable ASLR (PIE) on OS X (woodfairy)
  [#35704]
- **build**: reset embedder string to "-node.0" (Michaël Zasso)
  [#35700]
- **deps**: V8: cherry-pick 1648e050cade (Michaël Zasso)
  [#37587]
- **deps**: silence irrelevant V8 warnings (Michaël Zasso)
  [#37587]
- **deps**: fix V8 build issue with inline methods (Jiawen Geng)
  [#35415]
- **deps**: make v8.h compatible with VS2015 (Joao Reis)
  [#32116]
- **deps**: V8: forward declaration of `Rtl\*FunctionTable` (Refael
  Ackermann) [#32116]
- **deps**: V8: patch register-arm64.h (Refael Ackermann)
  [#32116]
- **deps**: V8: un-cherry-pick bd019bd (Refael Ackermann)
  [#32116]
- **deps**: update V8 to 9.0.257.11 (Michaël Zasso)
  [#37587]
- **deps**: bump minimum ICU version to 68 (Michaël Zasso)
  [#37330]
- **deps**: V8: cherry-pick 8957d4677aa7 (Michaël Zasso)
  [#37330]
- **deps**: V8: backport a11395433dbd (Michaël Zasso)
  [#37330]
- **deps**: V8: cherry-pick deb0813166f3 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick 9a6a22874c81 (Michaël Zasso)
  [#36139]
- **deps**: silence irrelevant V8 warning (Michaël Zasso)
  [#37330]
- **deps**: workaround stod() limitations on SmartOS (Colin Ihrig)
  [#37330]
- **deps**: fix V8 build issue with inline methods (Jiawen Geng)
  [#35415]
- **deps**: patch V8 to run on Xcode 8 (Mary Marchini)
  [#32116]
- **deps**: make v8.h compatible with VS2015 (Joao Reis)
  [#32116]
- **deps**: V8: forward declaration of `Rtl\*FunctionTable` (Refael
  Ackermann) [#32116]
- **deps**: V8: patch register-arm64.h (Refael Ackermann)
  [#32116]
- **deps**: patch V8 to run on older XCode versions (Ujjwal Sharma)
  [#32116]
- **deps**: V8: un-cherry-pick bd019bd (Refael Ackermann)
  [#32116]
- **deps**: update V8 to 8.9.255.19 (Michaël Zasso)
  [#37330]
- **deps**: V8: cherry-pick deb0813166f3 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick 9a6a22874c81 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick 2059ee813359 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick bde7ee5473d6 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick 9a712984025e (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick 0b96e5b0bfb2 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick fbb28902e049 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick 821fb3883a8e (Michaël Zasso)
  [#35700]
- **deps**: workaround stod() limitations on SmartOS (Colin Ihrig)
  [#36139]
- **deps**: fix V8 build issue with inline methods (Jiawen Geng)
  [#35415]
- **deps**: patch V8 to run on Xcode 8 (Mary Marchini)
  [#32116]
- **deps**: V8: silence irrelevant warnings (Michaël Zasso)
  [#32116]
- **deps**: make v8.h compatible with VS2015 (Joao Reis)
  [#32116]
- **deps**: V8: forward declaration of `Rtl\*FunctionTable` (Refael
  Ackermann) [#32116]
- **deps**: V8: patch register-arm64.h (Refael Ackermann)
  [#32116]
- **deps**: patch V8 to run on older XCode versions (Ujjwal Sharma)
  [#32116]
- **deps**: V8: un-cherry-pick bd019bd (Refael Ackermann)
  [#32116]
- **deps**: update V8 to 8.8.278.17 (Michaël Zasso)
  [#36139]
- **deps**: V8: cherry-pick 821fb3883a8e (Michaël Zasso)
  [#35700]
- **deps**: V8: cherry-pick 45e49775f5a3 (Michaël Zasso)
  [#35700]
- **deps**: V8: cherry-pick 7b3a27b7ae65 (Michaël Zasso)
  [#35700]
- **deps**: V8: cherry-pick d76abfed3512 (Michaël Zasso)
  [#35415]
- **deps**: fix V8 build issue with inline methods (Jiawen Geng)
  [#35415]
- **deps**: update V8 postmortem metadata script (Colin Ihrig)
  [#35415]
- **deps**: update V8 postmortem metadata script (Colin Ihrig)
  [#33579]
- **deps**: patch V8 to run on Xcode 8 (Mary Marchini)
  [#32116]
- **deps**: V8: silence irrelevant warnings (Michaël Zasso)
  [#32116]
- **deps**: make v8.h compatible with VS2015 (Joao Reis)
  [#32116]
- **deps**: V8: forward declaration of `Rtl\*FunctionTable` (Refael
  Ackermann) [#32116]
- **deps**: V8: patch register-arm64.h (Refael Ackermann)
  [#32116]
- **deps**: patch V8 to run on older XCode versions (Ujjwal Sharma)
  [#32116]
- **deps**: V8: un-cherry-pick bd019bd (Refael Ackermann)
  [#32116]
- **deps**: update V8 to 8.7.220 (Michaël Zasso)
  [#35700]
- **dns**: use url module instead of punycode for IDNA (Antoine du
  Hamel) [#35091]
- **doc**: update minimum supported Xcode to 11 (Michaël Zasso)
  [#37872]
- **doc**: update minimum supported GCC to 8.3 (Michaël Zasso)
  [#37871]
- **doc**: update AIX to GCC8 for v16.x (Ash Cripps)
  [#37677]
- **doc**: fixup http.IncomingMessage deprecation code (Guy Bedford)
  [#36917]
- **doc**: add http.IncomingMessage#connection (Pranshu Srivastava)
  [#33768]
- **events**: change EventTarget handler exception behavior (Nitzan
  Uziely) [#37237]
- **fs**: remove permissive rmdir recursive (Antoine du Hamel)
  [#37216]
- **fs**: add validation for fd and path (Dylan Elliott)
  [#35187]
- **fs**: runtime deprecate rmdir recursive option (Antoine du Hamel)
  [#37302]
- **fs**: fix flag and mode validation (James M Snell)
  [#37480]
- **http**: use objects with null prototype in Agent (Michaël Zasso)
  [#36409]
- **lib**: runtime deprecate access to process.binding('http_parser')
  (James M Snell) [#37813]
- **lib**: runtime deprecate access to process.binding('url') (James M
  Snell) [#37799]
- **lib**: make process.binding('util') return only type checkers (Anna
  Henningsen) [#37819]
- **lib**: runtime deprecate access to process.binding('crypto') (James
  M Snell) [#37790]
- **lib**: runtime deprecate access to process.binding('signal_wrap')
  (James M Snell) [#37800]
- **lib**: runtime deprecate access to process.binding('v8') (James M
  Snell) [#37789]
- **lib**: aggregate errors to avoid error swallowing (Antoine du Hamel)
  [#37460]
- **lib**: load v8_prof_processor dependencies as ESM (Michaël Zasso)
  [#37587]
- **lib**: runtime deprecate access to process.binding('async_wrap')
  (James M Snell) [#37576]
- **lib**: remove usage of url.parse (raisinten)
  [#36853]
- **lib**: add error handling for input stream (rexagod)
  [#31603]
- **lib,src**: update cluster to use Parent (Michael Dawson)
  [#36478]
- **module**: runtime deprecate subpath folder mappings (Antoine du
  Hamel) [#37215]
- **module**: runtime deprecate "main" index and extension lookups
  (Antoine du Hamel) [#37206]
- **module**: runtime deprecate invalid package.json main entries
  (Antoine du Hamel) [#37204]
- **module**: remove module.createRequireFromPath (Antoine du Hamel)
  [#37201]
- **module**: only set cache when finding module succeeds (Yongsheng
  Zhang) [#36642]
- **perf_hooks**: make performance a global (James M Snell)
  [#37970]
- **perf_hooks**: complete overhaul of the implementation (James M
  Snell) [#37136]
- **process**: disallow adding options to
  process.allowedNodeEnvironmentFlags (Antoine du Hamel)
  [#36660]
- **process**: runtime deprecate changing process.config (James M Snell)
  [#36902]
- **readline**: cursorTo throw error on NaN (Zijian Liu)
  [#36379]
- **src**: use non-deprecated GetCreationContext from V8 (Michaël Zasso)
  [#37587]
- **src**: remove V8_FT_ADAPTOR for V8 update (Colin Ihrig)
  [#37587]
- **src**: use non-deprecated V8 module APIs (Michaël Zasso)
  [#37587]
- **src**: update NODE_MODULE_VERSION to 93 (Michaël Zasso)
  [#37587]
- **src**: use non-deprecated V8 module and script APIs (Michaël Zasso)
  [#37330]
- **src**: update NODE_MODULE_VERSION to 92 (Michaël Zasso)
  [#37330]
- **src**: update NODE_MODULE_VERSION to 91 (Michaël Zasso)
  [#36139]
- **src**: mark internally exported functions as explicitly internal
  (Tyler Ang-Wanek) [#37000]
- **src**: inline AsyncCleanupHookHandle in headers (Tyler Ang-Wanek)
  [#37000]
- **src**: fix v8 api deprecation (Jiawen Geng)
  [#35700]
- **src**: update NODE_MODULE_VERSION to 90 (Michaël Zasso)
  [#35700]
- **src**: clean up embedder API (Anna Henningsen)
  [#35897]
- **test**: mark test-return-on-exit as flaky (Michaël Zasso)
  [#36139]
- **test**: mark WASI's test-return-on-exit as flaky (Colin Ihrig)
  [#36139]
- **tools**: update V8 gypfiles for 9.0 (Michaël Zasso)
  [#37587]
- **tools**: update V8 gypfiles for 8.9 (Michaël Zasso)
  [#37330]
- **tools**: update V8 gypfiles for 8.8 (Michaël Zasso)
  [#36139]
- **tools**: update V8 gypfiles for 8.7 (Michaël Zasso)
  [#35700]
- **worker**: send correct error status for worker init (Yash Ladha)
  [#36242]

PR-URL: #37678
BethGriggs added a commit that referenced this pull request Apr 20, 2021
Notable changes:

Deprecations and Removals:

- **(SEMVER-MAJOR)** **fs**: remove permissive rmdir recursive
  (Antoine du Hamel) [#37216]
- **(SEMVER-MAJOR)** **fs**: runtime deprecate rmdir recursive option
  (Antoine du Hamel) [#37302]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('http_parser') (James M Snell) [#37813]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('url') (James M Snell) [#37799]
- **(SEMVER-MAJOR)** **lib**: make process.binding('util') return only
  type checkers (Anna Henningsen) [#37819]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('crypto') (James M Snell) [#37790]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('signal_wrap') (James M Snell) [#37800]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('v8') (James M Snell) [#37789]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('async_wrap') (James M Snell) [#37576]
- **(SEMVER-MAJOR)** **module**: remove module.createRequireFromPath
  (Antoine du Hamel) [#37201]
- **(SEMVER-MAJOR)** **module**: runtime deprecate subpath folder
  mappings (Antoine du Hamel) [#37215]
- **(SEMVER-MAJOR)** **module**: runtime deprecate "main" index and
  extension lookups (Antoine du Hamel) [#37206]
- **(SEMVER-MAJOR)** **module**: runtime deprecate invalid package.json
  main entries (Antoine du Hamel) [#37204]
- **(SEMVER-MAJOR)** **process**: runtime deprecate changing
  process.config (James M Snell) [#36902]

Stable Timers Promises API:

The Timers Promises API provides an alternative set of timer functions
that return Promise objects. Added in Node.js v15.0.0, in this release
they graduate from experimental status to stable.

Contributed by James Snell - [#38112]

Toolchain and Compiler Upgrades:

Node.js v16.0.0 will be the first release where we ship prebuilt
binaries for Apple Silicon. While we’ll be providing separate tarballs
for the Intel (`darwin-x64`) and ARM (`darwin-arm64`) architectures the
macOS installer (`.pkg`) will be shipped as a ‘fat’ (multi-architecture)
binary.

- **(SEMVER-MAJOR)** **build**: remove support for Python 2 (Christian
  Clauss) [#36691]
- **(SEMVER-MAJOR)** **build**: default PYTHON to python3 in Makefile
  (Michaël Zasso) [#37764]
- **build**: update Makefile to support fat binary (Ash Cripps)
  [#37861]
- **(SEMVER-MAJOR)** **build**: enable ASLR (PIE) on OS X (woodfairy)
  [#35704]
- **build**: warn for gcc versions earlier than 8.3.0 (Richard Lau)
  [#37935]
- **(SEMVER-MAJOR)** **doc**: update minimum supported Xcode to 11
  (Michaël Zasso) [#37872]
- **(SEMVER-MAJOR)** **doc**: update minimum supported GCC to 8.3
  (Michaël Zasso) [#37871]
- **(SEMVER-MAJOR)** **doc**: update AIX to GCC8 for v16.x (Ash Cripps)
  [#37677]
- **tools**: set arch in Distribution.xml (Ash Cripps)
  [#38261]

V8 9.0:

The V8 JavaScript engine is updated to V8 9.0, including performance
tweaks and improvements.

This update also brings the ECMAScript RegExp Match Indices, which
provide the start and end indices of the captured string. The indices
array is available via the `.indices` property on match objects when the
regular expression has the `/d` flag.

Contributed by Michaël Zasso - [#37587]

Other Notable Changes:

- **(SEMVER-MINOR)** **assert**: graduate assert.match and
  assert.doesNotMatch (James M Snell) [#38111]
- **(SEMVER-MAJOR)** **buffer**: expose btoa and atob as globals
  (James M Snell) [#37786]
- **(SEMVER-MAJOR)** **deps**: bump minimum ICU version to 68
  (Michaël Zasso) [#37330]
- **deps**: update ICU to 69.1 (Michaël Zasso)
  [#38178]
- **deps**: update llhttp to 6.0.0 (Fedor Indutny)
  [#38277]
- **deps**: upgrade npm to 7.10.0 (Ruy Adorno)
  [#38254]
- **(SEMVER-MINOR)** **http**: add http.ClientRequest.getRawHeaderNames()
  (simov) [#37660]
- **(SEMVER-MAJOR)** **lib,src**: update cluster to use Parent
  (Michael Dawson) [#36478]
- **(SEMVER-MINOR)** **module**: add support for `node:`‑prefixed
  `require(…)` calls (ExE Boss) [#37246]
- **(SEMVER-MINOR)** **perf_hooks**: add histogram option to timerify
  (James M Snell) [#37475]
- **(SEMVER-MINOR)** **repl**: add auto‑completion for `node:`‑prefixed
  `require(…)` calls (ExE Boss) [#37246]
- **(SEMVER-MINOR)** **util**: add getSystemErrorMap() impl
  (eladkeyshawn) [#38101]

Semver-Major Commits:

- **(SEMVER-MAJOR)** **async_hooks**: add thisArg to AsyncResource.bind
  (James M Snell) [#36782]
- **(SEMVER-MAJOR)** **buffer**: expose btoa and atob as globals
  (James M Snell) [#37786]
- **(SEMVER-MAJOR)** **build**: remove support for Python 2
  (Christian Clauss) [#36691]
- **(SEMVER-MAJOR)** **build**: default PYTHON to python3 in Makefile
  (Michaël Zasso) [#37764]
- **(SEMVER-MAJOR)** **build**: update Makefile to support fat binary
  (Ash Cripps) [#37861]
- **(SEMVER-MAJOR)** **build**: include minimal V8 headers in
  distribution (Michaël Zasso) [#37570]
- **(SEMVER-MAJOR)** **build**: use C++11 ABI with libstdc++
  (Anna Henningsen) [#36634]
- **(SEMVER-MAJOR)** **build**: enable ASLR (PIE) on OS X (woodfairy)
  [#35704]
- **(SEMVER-MAJOR)** **deps**: update V8 to 9.0.257.11 (Michaël Zasso)
  [#37587]
- **(SEMVER-MAJOR)** **deps**: bump minimum ICU version to 68
  (Michaël Zasso) [#37330]
- **(SEMVER-MAJOR)** **deps**: update V8 to 8.9.255.19 (Michaël Zasso)
  [#37330]
- **(SEMVER-MAJOR)** **deps**: update V8 to 8.8.278.17 (Michaël Zasso)
  [#36139]
- **(SEMVER-MAJOR)** **deps**: update V8 to 8.7.220 (Michaël Zasso)
  [#35700]
- **(SEMVER-MAJOR)** **dns**: use url module instead of punycode for
  IDNA (Antoine du Hamel) [#35091]
- **(SEMVER-MAJOR)** **doc**: update minimum supported Xcode to 11
  (Michaël Zasso) [#37872]
- **(SEMVER-MAJOR)** **doc**: update minimum supported GCC to 8.3
  (Michaël Zasso) [#37871]
- **(SEMVER-MAJOR)** **doc**: update AIX to GCC8 for v16.x (Ash Cripps)
  [#37677]
- **(SEMVER-MAJOR)** **doc**: add http.IncomingMessage#connection
  (Pranshu Srivastava) [#33768]
- **(SEMVER-MAJOR)** **events**: change EventTarget handler exception
  behavior (Nitzan Uziely) [#37237]
- **(SEMVER-MAJOR)** **fs**: remove permissive rmdir recursive
  (Antoine du Hamel) [#37216]
- **(SEMVER-MAJOR)** **fs**: add validation for fd and path
  (Dylan Elliott) [#35187]
- **(SEMVER-MAJOR)** **fs**: runtime deprecate rmdir recursive option
  (Antoine du Hamel) [#37302]
- **(SEMVER-MAJOR)** **fs**: fix flag and mode validation
  (James M Snell) [#37480]
- **(SEMVER-MAJOR)** **http**: use objects with null prototype in Agent
  (Michaël Zasso) [#36409]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('http_parser') (James M Snell)
  [#37813]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('url') (James M Snell)
  [#37799]
- **(SEMVER-MAJOR)** **lib**: make process.binding('util') return only
  type checkers (Anna Henningsen) [#37819]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('crypto') (James M Snell) [#37790]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('signal_wrap') (James M Snell) [#37800]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('v8') (James M Snell) [#37789]
- **(SEMVER-MAJOR)** **lib**: aggregate errors to avoid error swallowing
  (Antoine du Hamel) [#37460]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('async_wrap') (James M Snell) [#37576]
- **(SEMVER-MAJOR)** **lib**: remove usage of url.parse
  (raisinten) [#36853]
- **(SEMVER-MAJOR)** **lib**: add error handling for input stream
  (rexagod) [#31603]
- **(SEMVER-MAJOR)** **lib,src**: update cluster to use Parent
  (Michael Dawson) [#36478]
- **(SEMVER-MAJOR)** **module**: runtime deprecate subpath folder
  mappings (Antoine du Hamel) [#37215]
- **(SEMVER-MAJOR)** **module**: runtime deprecate "main" index and
  extension lookups (Antoine du Hamel) [#37206]
- **(SEMVER-MAJOR)** **module**: runtime deprecate invalid package.json
  main entries (Antoine du Hamel) [#37204]
- **(SEMVER-MAJOR)** **module**: remove module.createRequireFromPath
  (Antoine du Hamel) [#37201]
- **(SEMVER-MAJOR)** **module**: only set cache when finding module
  succeeds (Yongsheng Zhang) [#36642]
- **(SEMVER-MAJOR)** **perf_hooks**: make performance a global (James M
  Snell) [#37970]
- **(SEMVER-MAJOR)** **perf_hooks**: complete overhaul of the
  implementation (James M Snell) [#37136]
- **(SEMVER-MAJOR)** **process**: disallow adding options to
  process.allowedNodeEnvironmentFlags (Antoine du Hamel)
  [#36660]
- **(SEMVER-MAJOR)** **process**: runtime deprecate changing
  process.config (James M Snell) [#36902]
- **(SEMVER-MAJOR)** **readline**: cursorTo throw error on NaN
  (Zijian Liu) [#36379]
- **(SEMVER-MAJOR)** **src**: mark internally exported functions as
  explicitly internal (Tyler Ang-Wanek) [#37000]
- **(SEMVER-MAJOR)** **src**: inline AsyncCleanupHookHandle in headers
  (Tyler Ang-Wanek) [#37000]
- **(SEMVER-MAJOR)** **src**: clean up embedder API (Anna Henningsen)
  [#35897]
- **(SEMVER-MAJOR)** **worker**: send correct error status for worker
  init (Yash Ladha) [#36242]

PR-URL: #37678
BethGriggs added a commit that referenced this pull request Apr 20, 2021
Notable changes:

Deprecations and Removals:

- **(SEMVER-MAJOR)** **fs**: remove permissive rmdir recursive
  (Antoine du Hamel) [#37216]
- **(SEMVER-MAJOR)** **fs**: runtime deprecate rmdir recursive option
  (Antoine du Hamel) [#37302]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('http_parser') (James M Snell) [#37813]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('url') (James M Snell) [#37799]
- **(SEMVER-MAJOR)** **lib**: make process.binding('util') return only
  type checkers (Anna Henningsen) [#37819]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('crypto') (James M Snell) [#37790]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('signal_wrap') (James M Snell) [#37800]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('v8') (James M Snell) [#37789]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('async_wrap') (James M Snell) [#37576]
- **(SEMVER-MAJOR)** **module**: remove module.createRequireFromPath
  (Antoine du Hamel) [#37201]
- **(SEMVER-MAJOR)** **module**: runtime deprecate subpath folder
  mappings (Antoine du Hamel) [#37215]
- **(SEMVER-MAJOR)** **module**: runtime deprecate "main" index and
  extension lookups (Antoine du Hamel) [#37206]
- **(SEMVER-MAJOR)** **module**: runtime deprecate invalid package.json
  main entries (Antoine du Hamel) [#37204]
- **(SEMVER-MAJOR)** **process**: runtime deprecate changing
  process.config (James M Snell) [#36902]

Stable Timers Promises API:

The Timers Promises API provides an alternative set of timer functions
that return Promise objects. Added in Node.js v15.0.0, in this release
they graduate from experimental status to stable.

Contributed by James Snell - [#38112]

Toolchain and Compiler Upgrades:

Node.js v16.0.0 will be the first release where we ship prebuilt
binaries for Apple Silicon. While we’ll be providing separate tarballs
for the Intel (`darwin-x64`) and ARM (`darwin-arm64`) architectures the
macOS installer (`.pkg`) will be shipped as a ‘fat’ (multi-architecture)
binary.

- **(SEMVER-MAJOR)** **build**: remove support for Python 2 (Christian
  Clauss) [#36691]
- **(SEMVER-MAJOR)** **build**: default PYTHON to python3 in Makefile
  (Michaël Zasso) [#37764]
- **build**: update Makefile to support fat binary (Ash Cripps)
  [#37861]
- **(SEMVER-MAJOR)** **build**: enable ASLR (PIE) on OS X (woodfairy)
  [#35704]
- **build**: warn for gcc versions earlier than 8.3.0 (Richard Lau)
  [#37935]
- **(SEMVER-MAJOR)** **doc**: update minimum supported Xcode to 11
  (Michaël Zasso) [#37872]
- **(SEMVER-MAJOR)** **doc**: update minimum supported GCC to 8.3
  (Michaël Zasso) [#37871]
- **(SEMVER-MAJOR)** **doc**: update AIX to GCC8 for v16.x (Ash Cripps)
  [#37677]
- **tools**: set arch in Distribution.xml (Ash Cripps)
  [#38261]

V8 9.0:

The V8 JavaScript engine is updated to V8 9.0, including performance
tweaks and improvements.

This update also brings the ECMAScript RegExp Match Indices, which
provide the start and end indices of the captured string. The indices
array is available via the `.indices` property on match objects when the
regular expression has the `/d` flag.

Contributed by Michaël Zasso - [#37587]

Other Notable Changes:

- **(SEMVER-MINOR)** **assert**: graduate assert.match and
  assert.doesNotMatch (James M Snell) [#38111]
- **(SEMVER-MAJOR)** **buffer**: expose btoa and atob as globals
  (James M Snell) [#37786]
- **(SEMVER-MAJOR)** **deps**: bump minimum ICU version to 68
  (Michaël Zasso) [#37330]
- **deps**: update ICU to 69.1 (Michaël Zasso)
  [#38178]
- **deps**: update llhttp to 6.0.0 (Fedor Indutny)
  [#38277]
- **deps**: upgrade npm to 7.10.0 (Ruy Adorno)
  [#38254]
- **(SEMVER-MINOR)** **http**: add http.ClientRequest.getRawHeaderNames()
  (simov) [#37660]
- **(SEMVER-MAJOR)** **lib,src**: update cluster to use Parent
  (Michael Dawson) [#36478]
- **(SEMVER-MINOR)** **module**: add support for `node:`‑prefixed
  `require(…)` calls (ExE Boss) [#37246]
- **(SEMVER-MINOR)** **perf_hooks**: add histogram option to timerify
  (James M Snell) [#37475]
- **(SEMVER-MINOR)** **repl**: add auto‑completion for `node:`‑prefixed
  `require(…)` calls (ExE Boss) [#37246]
- **(SEMVER-MINOR)** **util**: add getSystemErrorMap() impl
  (eladkeyshawn) [#38101]

Semver-Major Commits:

- **(SEMVER-MAJOR)** **async_hooks**: add thisArg to AsyncResource.bind
  (James M Snell) [#36782]
- **(SEMVER-MAJOR)** **buffer**: expose btoa and atob as globals
  (James M Snell) [#37786]
- **(SEMVER-MAJOR)** **build**: remove support for Python 2
  (Christian Clauss) [#36691]
- **(SEMVER-MAJOR)** **build**: default PYTHON to python3 in Makefile
  (Michaël Zasso) [#37764]
- **(SEMVER-MAJOR)** **build**: update Makefile to support fat binary
  (Ash Cripps) [#37861]
- **(SEMVER-MAJOR)** **build**: include minimal V8 headers in
  distribution (Michaël Zasso) [#37570]
- **(SEMVER-MAJOR)** **build**: use C++11 ABI with libstdc++
  (Anna Henningsen) [#36634]
- **(SEMVER-MAJOR)** **build**: enable ASLR (PIE) on OS X (woodfairy)
  [#35704]
- **(SEMVER-MAJOR)** **deps**: update V8 to 9.0.257.11 (Michaël Zasso)
  [#37587]
- **(SEMVER-MAJOR)** **deps**: bump minimum ICU version to 68
  (Michaël Zasso) [#37330]
- **(SEMVER-MAJOR)** **deps**: update V8 to 8.9.255.19 (Michaël Zasso)
  [#37330]
- **(SEMVER-MAJOR)** **deps**: update V8 to 8.8.278.17 (Michaël Zasso)
  [#36139]
- **(SEMVER-MAJOR)** **deps**: update V8 to 8.7.220 (Michaël Zasso)
  [#35700]
- **(SEMVER-MAJOR)** **dns**: use url module instead of punycode for
  IDNA (Antoine du Hamel) [#35091]
- **(SEMVER-MAJOR)** **doc**: update minimum supported Xcode to 11
  (Michaël Zasso) [#37872]
- **(SEMVER-MAJOR)** **doc**: update minimum supported GCC to 8.3
  (Michaël Zasso) [#37871]
- **(SEMVER-MAJOR)** **doc**: update AIX to GCC8 for v16.x (Ash Cripps)
  [#37677]
- **(SEMVER-MAJOR)** **doc**: add http.IncomingMessage#connection
  (Pranshu Srivastava) [#33768]
- **(SEMVER-MAJOR)** **events**: change EventTarget handler exception
  behavior (Nitzan Uziely) [#37237]
- **(SEMVER-MAJOR)** **fs**: remove permissive rmdir recursive
  (Antoine du Hamel) [#37216]
- **(SEMVER-MAJOR)** **fs**: add validation for fd and path
  (Dylan Elliott) [#35187]
- **(SEMVER-MAJOR)** **fs**: runtime deprecate rmdir recursive option
  (Antoine du Hamel) [#37302]
- **(SEMVER-MAJOR)** **fs**: fix flag and mode validation
  (James M Snell) [#37480]
- **(SEMVER-MAJOR)** **http**: use objects with null prototype in Agent
  (Michaël Zasso) [#36409]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('http_parser') (James M Snell)
  [#37813]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('url') (James M Snell)
  [#37799]
- **(SEMVER-MAJOR)** **lib**: make process.binding('util') return only
  type checkers (Anna Henningsen) [#37819]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('crypto') (James M Snell) [#37790]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('signal_wrap') (James M Snell) [#37800]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('v8') (James M Snell) [#37789]
- **(SEMVER-MAJOR)** **lib**: aggregate errors to avoid error swallowing
  (Antoine du Hamel) [#37460]
- **(SEMVER-MAJOR)** **lib**: runtime deprecate access to
  process.binding('async_wrap') (James M Snell) [#37576]
- **(SEMVER-MAJOR)** **lib**: remove usage of url.parse
  (raisinten) [#36853]
- **(SEMVER-MAJOR)** **lib**: add error handling for input stream
  (rexagod) [#31603]
- **(SEMVER-MAJOR)** **lib,src**: update cluster to use Parent
  (Michael Dawson) [#36478]
- **(SEMVER-MAJOR)** **module**: runtime deprecate subpath folder
  mappings (Antoine du Hamel) [#37215]
- **(SEMVER-MAJOR)** **module**: runtime deprecate "main" index and
  extension lookups (Antoine du Hamel) [#37206]
- **(SEMVER-MAJOR)** **module**: runtime deprecate invalid package.json
  main entries (Antoine du Hamel) [#37204]
- **(SEMVER-MAJOR)** **module**: remove module.createRequireFromPath
  (Antoine du Hamel) [#37201]
- **(SEMVER-MAJOR)** **module**: only set cache when finding module
  succeeds (Yongsheng Zhang) [#36642]
- **(SEMVER-MAJOR)** **perf_hooks**: make performance a global (James M
  Snell) [#37970]
- **(SEMVER-MAJOR)** **perf_hooks**: complete overhaul of the
  implementation (James M Snell) [#37136]
- **(SEMVER-MAJOR)** **process**: disallow adding options to
  process.allowedNodeEnvironmentFlags (Antoine du Hamel)
  [#36660]
- **(SEMVER-MAJOR)** **process**: runtime deprecate changing
  process.config (James M Snell) [#36902]
- **(SEMVER-MAJOR)** **readline**: cursorTo throw error on NaN
  (Zijian Liu) [#36379]
- **(SEMVER-MAJOR)** **src**: mark internally exported functions as
  explicitly internal (Tyler Ang-Wanek) [#37000]
- **(SEMVER-MAJOR)** **src**: inline AsyncCleanupHookHandle in headers
  (Tyler Ang-Wanek) [#37000]
- **(SEMVER-MAJOR)** **src**: clean up embedder API (Anna Henningsen)
  [#35897]
- **(SEMVER-MAJOR)** **worker**: send correct error status for worker
  init (Yash Ladha) [#36242]

PR-URL: #37678
@btsimonh
Copy link

thanks for the great work :).
For those of us who can't invest in M1, but want to self-build custom binaries, having the option to build for M1 on an x64 Intel box would be good in the future. If anyone has ideas on this, then can we drop additional notes here?

@rvagg
Copy link
Member

rvagg commented Apr 26, 2021

yes, this is possible but it's a super-hack for now, it needs to bubble up further through the toolchain and have some testing along the way

most of my notes are in nodejs/build#2474 (comment) and the thread below.

also nodejs/gyp-next#78 was added to support this in node-gyp - which is kind of a bypass rather than using gyp (essentially says "get out of the way and let my environment variables through").

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. build Issues and PRs related to build files or the CI. semver-major PRs that contain breaking changes and should be released in the next major version.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants