Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak for async/await with es6 target #36056

Closed
yahgwai opened this issue Jan 7, 2020 · 5 comments
Closed

Memory leak for async/await with es6 target #36056

yahgwai opened this issue Jan 7, 2020 · 5 comments
Assignees
Labels
Needs Investigation This issue needs a team member to investigate its status.

Comments

@yahgwai
Copy link

yahgwai commented Jan 7, 2020

When the supplied code is compiled with target es6 it results in a memory leak, however when the target is es2017 this is not the case.

TypeScript Version: 3.7.4
Node Version: 11.14.0

Search Terms:
memory leak
target
async
__awaiter

Code

class AsyncDoer {
    public async doIt() {
        return null;
    }
}

const run = async () => {
    for (let index = 0; index < 10000000; index++) {
        if(index % 100000 === 0) console.log(Math.floor(process.memoryUsage().heapUsed / 1000000), index);
        
        const doer = new AsyncDoer();
        await doer.doIt();
    }
}

run();

when compiled with es6 target results in:

var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {
    function adopt(value) { return value instanceof P ? value : new P(function (resolve) { resolve(value); }); }
    return new (P || (P = Promise))(function (resolve, reject) {
        function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
        function rejected(value) { try { step(generator["throw"](value)); } catch (e) { reject(e); } }
        function step(result) { result.done ? resolve(result.value) : adopt(result.value).then(fulfilled, rejected); }
        step((generator = generator.apply(thisArg, _arguments || [])).next());
    });
};
class AsyncDoer {
    doIt() {
        return __awaiter(this, void 0, void 0, function* () {
            return null;
        });
    }
}
const run = () => __awaiter(this, void 0, void 0, function* () {
    for (let index = 0; index < 10000000; index++) {
        if (index % 100000 === 0)
            console.log(Math.floor(process.memoryUsage().heapUsed / 1000000), index);
        const doer = new AsyncDoer();
        yield doer.doIt();
    }
});
run();

when compiled with es2017 target, results in:

class AsyncDoer {
    async doIt() {
        return null;
    }
}
const run = async () => {
    for (let index = 0; index < 10000000; index++) {
        if (index % 100000 === 0)
            console.log(Math.floor(process.memoryUsage().heapUsed / 1000000), index);
        const doer = new AsyncDoer();
        await doer.doIt();
    }
};
run();

Expected behavior:
When compiled with the following config:

{
    "compilerOptions": {
        "target": "es6",
    }
}

The js output can be run to completion using the command node --max-old-space-size=100.

Actual behavior:
The program crashes with an out of memory:

<--- Last few GCs --->

[1108:0x3dcd1f0]    16382 ms: Mark-sweep 49.5 (92.7) -> 49.5 (86.7) MB, 17.2 / 0.0 ms  (average mu = 0.744, current mu = 0.001) allocation failure GC in old space requested
[1108:0x3dcd1f0]    16400 ms: Mark-sweep 49.5 (86.7) -> 49.5 (55.7) MB, 18.0 / 0.0 ms  (average mu = 0.590, current mu = 0.001) last resort GC in old space requested
[1108:0x3dcd1f0]    16418 ms: Mark-sweep 49.5 (55.7) -> 49.5 (55.7) MB, 18.7 / 0.0 ms  (average mu = 0.412, current mu = 0.001) last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x31fa1cfcfc5d]
Security context: 0x37d53b59adf9 <JSObject>
    1: /* anonymous */ [0x164abf9127b9] [/home/chris/dev/mem_test/memTest.js:~17] [pc=0x31fa1d071475](this=0x2253745902f1 <Object map = 0x30ac8c082521>)
    2: next [0x37d53b596a79](this=0x225374584ac1 <JSGenerator>,0x38cb8e2022a1 <null>)
    3: fulfilled [0x2253745849e9] [/home/chris/dev/mem_test/memTest.js:~4] [pc=0x31fa1d06eaec](this=0x03e469c02331 <JSGlo...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: 0x95b8f0 node::Abort() [node]
 2: 0x95c836 node::OnFatalError(char const*, char const*) [node]
 3: 0xb3b77e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xb3b9b4 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xf3b002  [node]
 6: 0xf4b10f v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [node]
 7: 0xf14fa6 v8::internal::Factory::AllocateRawArray(int, v8::internal::PretenureFlag) [node]
 8: 0xf1aa2e v8::internal::Factory::CopyWeakArrayListAndGrow(v8::internal::Handle<v8::internal::WeakArrayList>, int, v8::internal::PretenureFlag) [node]
 9: 0x104f38a v8::internal::WeakArrayList::EnsureSpace(v8::internal::Isolate*, v8::internal::Handle<v8::internal::WeakArrayList>, int, v8::internal::PretenureFlag) [node]
10: 0x104f653 v8::internal::PrototypeUsers::Add(v8::internal::Isolate*, v8::internal::Handle<v8::internal::WeakArrayList>, v8::internal::Handle<v8::internal::Map>, int*) [node]
11: 0x1052e24 v8::internal::JSObject::LazyRegisterPrototypeUser(v8::internal::Handle<v8::internal::Map>, v8::internal::Isolate*) [node]
12: 0x10533c7 v8::internal::Map::GetOrCreatePrototypeChainValidityCell(v8::internal::Handle<v8::internal::Map>, v8::internal::Isolate*) [node]
13: 0xfabb34 v8::internal::LoadHandler::LoadFromPrototype(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::Handle<v8::internal::Smi>, v8::internal::MaybeObjectHandle, v8::internal::MaybeObjectHandle) [node]
14: 0xfb4ba4 v8::internal::LoadIC::ComputeHandler(v8::internal::LookupIterator*) [node]
15: 0xfbbfec v8::internal::LoadIC::UpdateCaches(v8::internal::LookupIterator*) [node]
16: 0xfbc6bc v8::internal::LoadIC::Load(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Name>) [node]
17: 0xfc0c05 v8::internal::Runtime_LoadIC_Miss(int, v8::internal::Object**, v8::internal::Isolate*) [node]
18: 0x31fa1cfcfc5d 
Aborted (core dumped)
@MartinJohns
Copy link
Contributor

Are you sure the memory is leaking, and you're not just running out of memory? As you're not awaiting any call you're creating 10.000.000 async calls simultaneously, and the polyfill requires a lot more memory than the natively supported version.

@yahgwai
Copy link
Author

yahgwai commented Jan 8, 2020

Apologies, the example should contain an await - missed that out. I've added the await back, and retested to confirm the same result. I've updated the example above.

@matthemsteger
Copy link

We were seeing this in production for our Apollo GraphQL implementation and for Node 12, believe the root cause is nodejs/node#30753 which will be fixed in 12.15.

@rbuckton
Copy link
Member

I have verified that @matthemsteger is correct. This was a bug in v8 which has since been fixed: https://bugs.chromium.org/p/v8/issues/detail?id=10031. I've verified this in v8 8.1.310:

// index.js
var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {
    function adopt(value) { return value instanceof P ? value : new P(function (resolve) { resolve(value); }); }
    return new (P || (P = Promise))(function (resolve, reject) {
        function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
        function rejected(value) { try { step(generator["throw"](value)); } catch (e) { reject(e); } }
        function step(result) { result.done ? resolve(result.value) : adopt(result.value).then(fulfilled, rejected); }
        step((generator = generator.apply(thisArg, _arguments || [])).next());
    });
};
class AsyncDoer {
    doIt() {
        return __awaiter(this, void 0, void 0, function* () {
            return null;
        });
    }
}
const run = () => __awaiter(this, void 0, void 0, function* () {
    for (let index = 0; index < 10000000; index++) {
        const doer = new AsyncDoer();
        yield doer.doIt();
    }
    print("done");
});
run();
> v8 -v
V8 version 8.1.310
> v8 index.js
done

@wyattzheng
Copy link

wyattzheng commented Oct 23, 2020

This is an annoying problem that i've been searching for days to find the solution, tsconfig.json use option {target:'es6'} ,then the generated code use __awaiter which implemented through es6 Generator instand of 'async & await',so this problem will come out if you use node 12 then use TypeScript to compile a async/await codes to es6,that's destructive. the solution of mine is set target to es7,then the problem is gone.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Investigation This issue needs a team member to investigate its status.
Projects
None yet
Development

No branches or pull requests

6 participants