[rfc] async version causes OOM (but not sync)

[yurydelendik]

• Version: v6.9.4
• Platform: Darwin Kernel Version 16.5.0
• Subsystem:

Based on mozilla/pdf.js#8419

STR:

1. clone https://github.com/yurydelendik/nodesegfault && npm install
   2.node pdf2svg.js

Expected result:
No node fatal error

Actual result:

...
<--- Last few GCs --->

   34764 ms: Mark-sweep 1149.4 (1215.2) -> 1148.8 (1190.2) MB, 14.0 / 0.0 ms [allocation failure] [GC in old space requested].
   34775 ms: Mark-sweep 1148.8 (1190.2) -> 1148.4 (1190.2) MB, 11.1 / 0.0 ms [allocation failure] [GC in old space requested].
   34794 ms: Mark-sweep 1148.4 (1190.2) -> 1148.4 (1156.2) MB, 18.9 / 0.0 ms [last resort gc].
   34802 ms: Mark-sweep 1148.4 (1156.2) -> 1148.4 (1156.2) MB, 7.7 / 0.0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x2684b5fcfb51 <JS Object>
    1: DoJoin(aka DoJoin) [native array.js:~129] [pc=0x45ab5998cec] (this=0x2684b5f04381 <undefined>,w=0x181e368c7f29 <JS Array[36]>,x=36,N=0x2684b5f043c1 <true>,J=0x2684b5f04411 <String[0]: >,I=0x2684b5fb46e1 <JS Function ConvertToString (SharedFunctionInfo 0x2684b5f52dc9)>)
    2: Join(aka Join) [native array.js:180] [pc=0x45ab590e3b2] (this=0x2684b5f04381 <undefined>,w=0x181e368c7f29 <JS Arr...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/opt/local/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/opt/local/bin/node]
 3: v8::Utils::ReportApiFailure(char const*, char const*) [/opt/local/bin/node]
 4: v8::Utils::ApiCheck(bool, char const*, char const*) [/opt/local/bin/node]
 5: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/opt/local/bin/node]
 6: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/opt/local/bin/node]
 7: v8::internal::Runtime_StringBuilderConcat(int, v8::internal::Object**, v8::internal::Isolate*) [/opt/local/bin/node]
 8: 0x45ab54092a7
Abort trap: 6

Works fine when sync versions of writeFile or mkdir used (see mozilla/pdf.js#8461)

[sam-github]

Node didn't segfault, it aborted because it ran out of memory

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

[mscdex]

General help questions should be posted to the nodejs/help issue tracker instead.

[bnoordhuis]

I appreciate that you took the time to put together a test case but I don't see anything that indicates a bug in node.js core, just a memory leak in the application. Closing, can reopen if that assessment is incorrect.

[yurydelendik]

New finding: using non-native Node.js promises fixes the issue as well, e.g. global.Promise = undefined; forces to use PDF.js polyfill. This means there is no critical OOM leak on PDF.js side.

See also nodejs/help#652 and yurydelendik/nodesegfault@2bb6f26

[bnoordhuis]

That doesn't necessarily follow, it depends on what the third-party promises library does differently from native promises. If it's bluebird, see #6673 (comment).

[yurydelendik]

it depends on what the third-party promises library does differently from native promises.

Promise of promises that their behavior will be alike. PDF.js does not use bluebird, it uses its own implemented by spec polyfill (and this polyfill probably uses much more memory than native promises).

(On different note, this issue was closed on the statement that PDF.js is not using memory efficiently, but does not provide explanation why sync versions of writing to file does not cause OOM, but async does)

[bnoordhuis]

For posterity, turns out it wasn't a node.js issue: mozilla/pdf.js#8461 (comment)