Buffer Memory Leak

[trevnorris]

I realized this while reviewing changes I want to make discussed in the mailing list.

Fast Buffers have a performance advantage since they are allocated from a single larger buffer. But that means if there is even one byte hanging around, the entire SlowBuffer must hang around in memory. The following is a script that will prove the point:

// WARNING: This script will crash your computer!
// ran this with `./out/Release/node --expose-gc mem-test.js
const SIZE = Buffer.poolSize - 2;
// if you want to test this w/o crashing your computer, reduce ITTR
const ITTR = 1e6;

var SlowBuffer = require('buffer').SlowBuffer;
var store = [];
var rss, tmp, i;

rss = process.memoryUsage().rss;

for (i = 0; i < ITTR; i++) {
// was using this as a control
    //tmp = SlowBuffer(SIZE);

    store.push(Buffer(1));
// comment out the following and you'll see that the problem no
// longer exists
    tmp = Buffer(SIZE);

    if (i % 1e3 === 0) gc();
}


setTimeout(function() {
    var nr = process.memoryUsage().rss
    console.log(((nr - rss) / 1024 / 1024).toFixed(2));
}, 1000);

Basically every loop will allocate
Buffer.poolSize
memory instead of just 1. I'll write an http example later that exhibits the same behavior.

[bnoordhuis]

I appreciate that you bring it up but I'm well aware of that. It's not a bug, it's a conscious time/space trade-off.

Malloc'ing memory for each buffer independently has a massive performance impact. It's exactly the reason why the current scheme was implemented.

[trevnorris]

@bnoordhuis I understand the implications of malloc'ing for each smaller buffer, but since Buffers are a major part of node, it seems there is a better memory allocation algorithm. You understand how memory pools are currently created, and that something like the following would leave allocated but unused memory hanging around:

var a = Buffer(1);
var b = Buffer(Buffer.poolSize / 2);
// here a new SlowBuffer is allocated and the old pool is forgotten about,
// but only about half of it was used.
var c = Buffer(Buffer.poolSize / 2);

I'm not sure it will help, but I'm investigating how v8 creates and tracks Strings. I want to understand how v8 can track many small strings, and still gc them when no longer used. Thinking is this could help the issue.

[bnoordhuis]

I'm not sure it will help, but I'm investigating how v8 creates and tracks Strings.

With the same garbage collector that tracks buffers. But strings and buffers are different beasts. Strings don't have backlinks to other strings, buffers do.

[piscisaureus]

With the same garbage collector that tracks buffers. But strings and buffers are different beasts. Strings don't have backlinks to other strings, buffers do.

Well, eh... sliced strings :-)

[trevnorris]

@bnoordhuis I wrote a benchmark test to try and accurately determine memory requirements and initialization speeds between strings and Buffers. Each one of the for() loops in the gist was run separately to get accurate measurements.

$ ./out/Release/node --expose-gc str-test.js
SlowBuffer: 4063ms
329.24 MB
SlowBuffer SIZE: 1754ms
299.32 MB
Buffer: 1856ms
137.12 MB
Buffer SIZE: 453ms
83.75 MB
String: 562ms
122.24 MB
Native: 407ms
92.84 MB

The two I feel are most relevant are Buffer and String. While Buffer SIZE is slightly faster and uses less memory than String it only allocates the memory and doesn't actually fill it. Also the poolSize for Buffer was pre-allocated to the exact size of the incoming data.

Basically what I'm getting at is there must be a faster and garbage collectible way to allocate memory for Buffers. As far as the problem of tracking memory pointers of Buffer slices, that could be mitigated down to the buf.slice() command. Instead of happening on all Buffers that are created within the poolSize.

I'm still investigating exactly how v8 does it for strings, and hoping to come across something we can use.

@piscisaureus Following through the logic in v8/src/heap.cc#3455-3546, FLAG_string_slices is only available when staring with the option --string_slices. Memory is copied otherwise.

[bnoordhuis]

For posterity: I do have some ideas on how to deal with the issue of buffer retention.

To recap, the reason we have Buffers and SlowBuffers is that it's:

1. Expensive to malloc() memory, and
2. Expensive to have many objects that need finalization (i.e. have WeakCallbacks).

That's why we create a few big SlowBuffers (expensive) and then slice them up in many small Buffers (cheap because they're regular JS objects).

What to me seems like the most straightforward solution, is to extend V8 to support buffers natively. V8 has a compacting garbage collector so you probably don't want to allocate the buffer's memory on the JS heap (because objects get copied around all the time) but either malloc() it or allocate it in a separate zone (i.e. heap) that is not compacted.

The disadvantage of malloc() is that it can be slow. The disadvantage of a separate zone is that you need to implement something like malloc() on top in order to reclaim memory.

It's probably easiest to start with malloc() and if that doesn't pan out, switch to the zone-based solution.