-
Notifications
You must be signed in to change notification settings - Fork 29.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cli: allow running wasm in limited vmemory with --disable-wasm-trap-handler #52766
Conversation
Review requested:
|
A more real world example of what this PR fixes:
(This happens because Webpack uses a wasm implementation of md4) |
1906f18
to
61b7375
Compare
The output of `node -p process.config.variables.asan` includes a newline character so it's never exactly "1", which means asan is always "off" for the status files. This fixes the detection by stripping whitespaces from the output.
…andler By default, Node.js enables trap-handler-based WebAssembly bound checks. As a result, V8 does not need to insert inline bound checks int the code compiled from WebAssembly which may speedup WebAssembly execution significantly, but this optimization requires allocating a big virtual memory cage (currently 10GB). If the Node.js process does not have access to a large enough virtual memory address space due to system configurations or hardware limitations, users won't be able to run any WebAssembly that involves allocation in this virtual memory cage and will see an out-of-memory error. ```console $ ulimit -v 5000000 $ node -p "new WebAssembly.Memory({ initial: 10, maximum: 100 });" [eval]:1 new WebAssembly.Memory({ initial: 10, maximum: 100 }); ^ RangeError: WebAssembly.Memory(): could not allocate memory at [eval]:1:1 at runScriptInThisContext (node:internal/vm:209:10) at node:internal/process/execution:118:14 at [eval]-wrapper:6:24 at runScript (node:internal/process/execution:101:62) at evalScript (node:internal/process/execution:136:3) at node:internal/main/eval_string:49:3 ``` `--disable-wasm-trap-handler` disables this optimization so that users can at least run WebAssembly (with a less optimial performance) when the virtual memory address space available to their Node.js process is lower than what the V8 WebAssembly memory cage needs.
61b7375
to
a087263
Compare
cc @nodejs/cpp-reviewers @nodejs/build |
Added YAML and manpage entries, thanks for the reminder. As for stability index, it seems unnecessary for a flag that serves as an escape hatch in a special environment - we don't usually add stability index to this kind of flags, either, and there is no clear criteria when this should be considered stable or how it could change over time - I expect the flag to just keep working as it is. If we must assign an index, I think it's fine to just mark it stable. But I'd prefer to just leave it out like most other optional CLI flags. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR-URL: #52766 Reviewed-By: Rafael Gonzaga <[email protected]>
Notable changes: doc: * add pimterry to collaborators (Tim Perry) #52874 inspector: * (SEMVER-MINOR) introduce the `--inspect-wait` flag (Kohei Ueno) #52734 test_runner: * (SEMVER-MINOR) support test plans (Colin Ihrig) #52860 tools: * (SEMVER-MINOR) fix get_asan_state() in tools/test.py (Joyee Cheung) #52766 * (SEMVER-MINOR) support max_virtual_memory test configuration (Joyee Cheung) #52766 * (SEMVER-MINOR) support != in test status files (Joyee Cheung) #52766 zlib: * (SEMVER-MINOR) expose zlib.crc32() (Joyee Cheung) #52692 PR-URL: #53486
By default, Node.js enables trap-handler-based WebAssembly bound checks. As a result, V8 does not need to insert inline bound checks int the code compiled from WebAssembly which may speedup WebAssembly execution significantly, but this optimization requires allocating a big virtual memory cage (currently 10GB). If the Node.js process does not have access to a large enough virtual memory address space due to system configurations or hardware limitations, users won't be able to run any WebAssembly that involves allocation in this virtual memory cage and will see an out-of-memory error. ```console $ ulimit -v 5000000 $ node -p "new WebAssembly.Memory({ initial: 10, maximum: 100 });" [eval]:1 new WebAssembly.Memory({ initial: 10, maximum: 100 }); ^ RangeError: WebAssembly.Memory(): could not allocate memory at [eval]:1:1 at runScriptInThisContext (node:internal/vm:209:10) at node:internal/process/execution:118:14 at [eval]-wrapper:6:24 at runScript (node:internal/process/execution:101:62) at evalScript (node:internal/process/execution:136:3) at node:internal/main/eval_string:49:3 ``` `--disable-wasm-trap-handler` disables this optimization so that users can at least run WebAssembly (with a less optimial performance) when the virtual memory address space available to their Node.js process is lower than what the V8 WebAssembly memory cage needs. PR-URL: #52766 Reviewed-By: Rafael Gonzaga <[email protected]>
Notable changes: doc: * add pimterry to collaborators (Tim Perry) #52874 inspector: * (SEMVER-MINOR) introduce the `--inspect-wait` flag (Kohei Ueno) #52734 test_runner: * (SEMVER-MINOR) support test plans (Colin Ihrig) #52860 tools: * (SEMVER-MINOR) fix get_asan_state() in tools/test.py (Joyee Cheung) #52766 * (SEMVER-MINOR) support max_virtual_memory test configuration (Joyee Cheung) #52766 * (SEMVER-MINOR) support != in test status files (Joyee Cheung) #52766 zlib: * (SEMVER-MINOR) expose zlib.crc32() (Joyee Cheung) #52692 PR-URL: #53486
Notable changes: doc: * add pimterry to collaborators (Tim Perry) #52874 inspector: * (SEMVER-MINOR) introduce the `--inspect-wait` flag (Kohei Ueno) #52734 test_runner: * (SEMVER-MINOR) support test plans (Colin Ihrig) #52860 tools: * (SEMVER-MINOR) fix get_asan_state() in tools/test.py (Joyee Cheung) #52766 * (SEMVER-MINOR) support max_virtual_memory test configuration (Joyee Cheung) #52766 * (SEMVER-MINOR) support != in test status files (Joyee Cheung) #52766 zlib: * (SEMVER-MINOR) expose zlib.crc32() (Joyee Cheung) #52692 PR-URL: #53486
Notable changes: doc: * add pimterry to collaborators (Tim Perry) #52874 inspector: * (SEMVER-MINOR) introduce the `--inspect-wait` flag (Kohei Ueno) #52734 test_runner: * (SEMVER-MINOR) support test plans (Colin Ihrig) #52860 tools: * (SEMVER-MINOR) fix get_asan_state() in tools/test.py (Joyee Cheung) #52766 * (SEMVER-MINOR) support max_virtual_memory test configuration (Joyee Cheung) #52766 * (SEMVER-MINOR) support != in test status files (Joyee Cheung) #52766 zlib: * (SEMVER-MINOR) expose zlib.crc32() (Joyee Cheung) #52692 PR-URL: #53486
PR-URL: nodejs#52766 Reviewed-By: Rafael Gonzaga <[email protected]>
PR-URL: nodejs#52766 Reviewed-By: Rafael Gonzaga <[email protected]>
The output of `node -p process.config.variables.asan` includes a newline character so it's never exactly "1", which means asan is always "off" for the status files. This fixes the detection by stripping whitespaces from the output. PR-URL: nodejs#52766 Reviewed-By: Rafael Gonzaga <[email protected]>
By default, Node.js enables trap-handler-based WebAssembly bound checks. As a result, V8 does not need to insert inline bound checks int the code compiled from WebAssembly which may speedup WebAssembly execution significantly, but this optimization requires allocating a big virtual memory cage (currently 10GB). If the Node.js process does not have access to a large enough virtual memory address space due to system configurations or hardware limitations, users won't be able to run any WebAssembly that involves allocation in this virtual memory cage and will see an out-of-memory error. ```console $ ulimit -v 5000000 $ node -p "new WebAssembly.Memory({ initial: 10, maximum: 100 });" [eval]:1 new WebAssembly.Memory({ initial: 10, maximum: 100 }); ^ RangeError: WebAssembly.Memory(): could not allocate memory at [eval]:1:1 at runScriptInThisContext (node:internal/vm:209:10) at node:internal/process/execution:118:14 at [eval]-wrapper:6:24 at runScript (node:internal/process/execution:101:62) at evalScript (node:internal/process/execution:136:3) at node:internal/main/eval_string:49:3 ``` `--disable-wasm-trap-handler` disables this optimization so that users can at least run WebAssembly (with a less optimial performance) when the virtual memory address space available to their Node.js process is lower than what the V8 WebAssembly memory cage needs. PR-URL: nodejs#52766 Reviewed-By: Rafael Gonzaga <[email protected]>
Notable changes: cli: * (SEMVER-MINOR) allow running wasm in limited vmem with --disable-wasm-trap-handler (Joyee Cheung) nodejs#52766 doc: * add pimterry to collaborators (Tim Perry) nodejs#52874 fs: * (SEMVER-MINOR) allow 'withFileTypes' to be used with globs (Aviv Keller) nodejs#52837 inspector: * (SEMVER-MINOR) introduce the `--inspect-wait` flag (Kohei Ueno) nodejs#52734 lib,src: * remove --experimental-policy (Rafael Gonzaga) nodejs#52583 perf_hooks: * (SEMVER-MINOR) add `deliveryType` and `responseStatus` fields (Matthew Aitken) nodejs#51589 test_runner: * (SEMVER-MINOR) support test plans (Colin Ihrig) nodejs#52860 zlib: * (SEMVER-MINOR) expose zlib.crc32() (Joyee Cheung) nodejs#52692 PR-URL: nodejs#52971
Notable changes: doc: * add pimterry to collaborators (Tim Perry) nodejs#52874 inspector: * (SEMVER-MINOR) introduce the `--inspect-wait` flag (Kohei Ueno) nodejs#52734 test_runner: * (SEMVER-MINOR) support test plans (Colin Ihrig) nodejs#52860 tools: * (SEMVER-MINOR) fix get_asan_state() in tools/test.py (Joyee Cheung) nodejs#52766 * (SEMVER-MINOR) support max_virtual_memory test configuration (Joyee Cheung) nodejs#52766 * (SEMVER-MINOR) support != in test status files (Joyee Cheung) nodejs#52766 zlib: * (SEMVER-MINOR) expose zlib.crc32() (Joyee Cheung) nodejs#52692 PR-URL: nodejs#53486
PR-URL: nodejs#52766 Reviewed-By: Rafael Gonzaga <[email protected]>
PR-URL: nodejs#52766 Reviewed-By: Rafael Gonzaga <[email protected]>
The output of `node -p process.config.variables.asan` includes a newline character so it's never exactly "1", which means asan is always "off" for the status files. This fixes the detection by stripping whitespaces from the output. PR-URL: nodejs#52766 Reviewed-By: Rafael Gonzaga <[email protected]>
By default, Node.js enables trap-handler-based WebAssembly bound checks. As a result, V8 does not need to insert inline bound checks int the code compiled from WebAssembly which may speedup WebAssembly execution significantly, but this optimization requires allocating a big virtual memory cage (currently 10GB). If the Node.js process does not have access to a large enough virtual memory address space due to system configurations or hardware limitations, users won't be able to run any WebAssembly that involves allocation in this virtual memory cage and will see an out-of-memory error. ```console $ ulimit -v 5000000 $ node -p "new WebAssembly.Memory({ initial: 10, maximum: 100 });" [eval]:1 new WebAssembly.Memory({ initial: 10, maximum: 100 }); ^ RangeError: WebAssembly.Memory(): could not allocate memory at [eval]:1:1 at runScriptInThisContext (node:internal/vm:209:10) at node:internal/process/execution:118:14 at [eval]-wrapper:6:24 at runScript (node:internal/process/execution:101:62) at evalScript (node:internal/process/execution:136:3) at node:internal/main/eval_string:49:3 ``` `--disable-wasm-trap-handler` disables this optimization so that users can at least run WebAssembly (with a less optimial performance) when the virtual memory address space available to their Node.js process is lower than what the V8 WebAssembly memory cage needs. PR-URL: nodejs#52766 Reviewed-By: Rafael Gonzaga <[email protected]>
Notable changes: cli: * (SEMVER-MINOR) allow running wasm in limited vmem with --disable-wasm-trap-handler (Joyee Cheung) nodejs#52766 doc: * add pimterry to collaborators (Tim Perry) nodejs#52874 fs: * (SEMVER-MINOR) allow 'withFileTypes' to be used with globs (Aviv Keller) nodejs#52837 inspector: * (SEMVER-MINOR) introduce the `--inspect-wait` flag (Kohei Ueno) nodejs#52734 lib,src: * remove --experimental-policy (Rafael Gonzaga) nodejs#52583 perf_hooks: * (SEMVER-MINOR) add `deliveryType` and `responseStatus` fields (Matthew Aitken) nodejs#51589 test_runner: * (SEMVER-MINOR) support test plans (Colin Ihrig) nodejs#52860 zlib: * (SEMVER-MINOR) expose zlib.crc32() (Joyee Cheung) nodejs#52692 PR-URL: nodejs#52971
Notable changes: doc: * add pimterry to collaborators (Tim Perry) nodejs#52874 inspector: * (SEMVER-MINOR) introduce the `--inspect-wait` flag (Kohei Ueno) nodejs#52734 test_runner: * (SEMVER-MINOR) support test plans (Colin Ihrig) nodejs#52860 tools: * (SEMVER-MINOR) fix get_asan_state() in tools/test.py (Joyee Cheung) nodejs#52766 * (SEMVER-MINOR) support max_virtual_memory test configuration (Joyee Cheung) nodejs#52766 * (SEMVER-MINOR) support != in test status files (Joyee Cheung) nodejs#52766 zlib: * (SEMVER-MINOR) expose zlib.crc32() (Joyee Cheung) nodejs#52692 PR-URL: nodejs#53486
* chore: bump node in DEPS to v20.15.0 * doc: Add OpenSSL errors to API docs nodejs/node#34213 * test: crypto-rsa-dsa testing for dynamic openssl nodejs/node#52781 * src: allow preventing debug signal handler start nodejs/node#46681 * cli: allow running wasm in limited vmemory with --disable-wasm-trap-handler nodejs/node#52766 * chore: fixup indices --------- Co-authored-by: electron-roller[bot] <84116207+electron-roller[bot]@users.noreply.github.com> Co-authored-by: Shelley Vohr <[email protected]>
* chore: bump node in DEPS to v20.15.0 * doc: Add OpenSSL errors to API docs nodejs/node#34213 * test: crypto-rsa-dsa testing for dynamic openssl nodejs/node#52781 * src: allow preventing debug signal handler start nodejs/node#46681 * cli: allow running wasm in limited vmemory with --disable-wasm-trap-handler nodejs/node#52766 * chore: fixup indices --------- Co-authored-by: electron-roller[bot] <84116207+electron-roller[bot]@users.noreply.github.com> Co-authored-by: Shelley Vohr <[email protected]>
* chore: bump node in DEPS to v20.15.0 * doc: Add OpenSSL errors to API docs nodejs/node#34213 * test: crypto-rsa-dsa testing for dynamic openssl nodejs/node#52781 * src: allow preventing debug signal handler start nodejs/node#46681 * cli: allow running wasm in limited vmemory with --disable-wasm-trap-handler nodejs/node#52766 * chore: fixup patch indices * chore: update patches --------- Co-authored-by: electron-roller[bot] <84116207+electron-roller[bot]@users.noreply.github.com> Co-authored-by: Shelley Vohr <[email protected]> Co-authored-by: PatchUp <73610968+patchup[bot]@users.noreply.github.com>
* chore: bump node in DEPS to v20.15.0 * doc: Add OpenSSL errors to API docs nodejs/node#34213 * test: crypto-rsa-dsa testing for dynamic openssl nodejs/node#52781 * src: allow preventing debug signal handler start nodejs/node#46681 * cli: allow running wasm in limited vmemory with --disable-wasm-trap-handler nodejs/node#52766 * chore: fixup patch indices * chore: update patches --------- Co-authored-by: electron-roller[bot] <84116207+electron-roller[bot]@users.noreply.github.com> Co-authored-by: Shelley Vohr <[email protected]> Co-authored-by: PatchUp <73610968+patchup[bot]@users.noreply.github.com>
tools: support != in test status files
tools: support max_virtual_memory test configuration
tools: fix get_asan_state() in tools/test.py
The output of
node -p process.config.variables.asan
includesa newline character so it's never exactly "1", which means
asan is always "off" for the status files. This fixes the
detection by stripping whitespaces from the output.
cli: allow running wasm in limited vmemory with --disable-wasm-trap-handler
By default, Node.js enables trap-handler-based WebAssembly bound
checks. As a result, V8 does not need to insert inline bound checks
int the code compiled from WebAssembly which may speedup WebAssembly
execution significantly, but this optimization requires allocating
a big virtual memory cage (currently 10GB). If the Node.js process
does not have access to a large enough virtual memory address space
due to system configurations or hardware limitations, users won't
be able to run any WebAssembly that involves allocation in this
virtual memory cage and will see an out-of-memory error.
--disable-wasm-trap-handler
disables this optimization so thatusers can at least run WebAssembly (with a less optimial performance)
when the virtual memory address space available to their Node.js
process is lower than what the V8 WebAssembly memory cage needs.
Background: https://docs.google.com/document/u/3/d/1PM4Zqmlt8ac5O8UNQfY7fOsem-6MhbsB-vjFI-9XK6w/edit#heading=h.diogznvalour