Parse Server memory usage

[benishak]

Env:

Parse Server 2.2.16
Parse SDK 1.9.0
AWS

Problem

Consider this code run on the Client

var Query = new Parse.Query("Books");
Query.limit(10000);
Query.find().then(
   function(books){
         // do something with books
        books = [] // clean up RAM
   }
)

in this example the moment the result is returned to the client, the memory at the client side hike to 800MB, but when the line books = [] execute the memory usage back to normal around 70MB.

The situation in Parse Server (server side) : the moment I request the result, the memory usage hikes also to 1GB in this example but the difference is that it never goes back to normal till I have to restart the node.

Is there any workaround to fix such issue?

[hramos]

Does this happen on version 2.2.17, which is the latest as of this issue's creation?

[benishak]

Yes

[joeyslack]

Also seeing huge memory spikes.

[benishak]

this could be a problem with node itself because of a bug in V8 Engine, I was using v4.4.2, now I upgraded to v4.5.0 and will come back later with results

source: https://nodejs.org/en/blog/vulnerability/june-2016-security-releases/

[benishak]

After running for a week I can say it was NodeJS issue because of the old V8 CG bug in node < 4.5
I'm now using node 6.4 in cluster mode, things are acceptable,

But the top example still causing a memory leak,
I install memwatch-next and inside the callback of leak event I put some logs and i got the following

"growth":9919040,"reason":"heap growth over 5 consecutive GCs (1m 2s) - 549.26 mb/hr"}

I still not sure what cause this leak, but I disable all our cloud code to make sure that it is not because of our code ...

A work around was to force CG to run when the leak happen so I put something like this

memwatch.on('leak', function(info) { memwatch.gc() });

memwatch.gc() will force CG to run immidiately instead of waiting it to run by itself

[flovilmart]

A workaround is nice but a fix is better. Do you want to tackle that issue?

[benishak]

@hramos @flovilmart I think the issue turns out to be related to Cache somehow, after switching to Redis Cache, this become ok, but I'm still testing ....

[flovilmart]

Alirght :) flushing the cash would be nice indeed, or using a LRU cache or with timeout would be nice too.

[benishak]

redis fixed this issue