Linux ARM : --max-old-space-size flag not working as expected

[jcommaret]

Version

v22.11.0

Platform

Linux jco-VMware20-1 6.8.0-48-generic #48-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep 27 14:35:45 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

Sorry, but the bug needs a large amount of code to crash, so here is what i did :

Clone Microsoft visual studio code repo :

1. git clone [email protected]:microsoft/vscode.git
2. cd vscode
3. set the --max-old-space-size to a lower amount of RAM (3000 in my case)
4. npm install
5. npm run watch
6. wait for the crash : (error 137)
7. Cry

How often does it reproduce? Is there a required condition?

Allways reproducing

What is the expected behavior? Why is that the expected behavior?

The code should not kill nodejs with error 137, because this is an Out of memory error, but the setup is made to take in account there is only 3000.

I was able to have the same code working on macOS ARM with lower ram configuration (3000 RAM)

What do you see instead?

error 137

Additional information

I wonder if there is no security issue about that.

--max-old-space-size flag have impact on memory management.

Here is the conversation with Microsoft VSCode Team, when i told them this command should work
microsoft/vscode#232535

[RedYetiDev]

Are you sure this isn't a bug with VSCode? It's possible their code needs more memory than your allowing it to access

[jcommaret]

@RedYetiDev : --max-old-space-size is a flag to limit memory usage. https://nodejs.org/api/cli.html#--max-old-space-sizesize-in-mib

The same flag is working with 3000mb on my Mac.

[RedYetiDev]

Can you reproduce outside of VSCode?

[jcommaret]

@RedYetiDev : I don't have another project who require that amount of ram.

[RedYetiDev]

IMO there's a good change that VSCode requires more memory than you're allowing it to have. Despite it working on MacOS, it's possible the memory management of the two systems is much different.

I'm happy to get a second opinion on this tho.

[joebowbeer]

set the --max-old-space-size to a lower amount of RAM (3000 in my case)

How are you setting max-old-space-size?
What version of node are you running?

In general, each app has a memory "working set" that may depend on the platform configuration, and the max old space size needs to be configured to be larger than the working set size, or the app crashes.

By default, the max old space size is configured based on the memory limit. What is the memory limit in your case?

[jcommaret]

set the --max-old-space-size to a lower amount of RAM (3000 in my case)

How are you setting max-old-space-size? What version of node are you running?

I am modifying the 3 values of the package.json, I have used 20.18.0, and I also updated to 22.11.0 to check this was not an old version problem solved since, both worked on macOS, both is crashing on Linux.

In general, each app has a memory "working set" that may depend on the platform configuration, and the max old space size needs to be configured to be larger than the working set size, or the app crashes.

By default, the max old space size is configured based on the memory limit. What is the memory limit in your case?

What I understand, is that there is a memory sweet-spot to find.
I able to have it working at ram set at 2048 on macOS and even with 5136 on Linux, this doesn't work ?
This is 2,5 x more ram allocated, and in my opinion, the exact same command should involve the exact same output.
This is why I think this is a bug on how nodejs on Linux ARM is managing the --max-old-space-size=

[bnoordhuis]

137 means node was killed by a SIGKILL signal, probably OOM-killed by the kernel. Check dmesg and lower --max-old-space-size. You're trying to use more memory than the system will let you.

Local issue, not a bug. Closing.

[jcommaret]

@bnoordhuis : --max-old-space-size is made to limit memory usage.
The system should not allow me to use more memory because of the flag.
I don't understand what you don't get.

MacOS Arm : task work with 2048mo
Linux ARM : task is always crashing even with 5096 mo.

The instructions are the same.

Why linux is crashing ?

[bnoordhuis]

You misunderstand. You instruct node to use up to 3 GB but linux won't let you.

[jcommaret]

You misunderstand. You instruct node to use up to 3 GB but linux won't let you.

Why ? And why the same task is working with MacOS ?

[joebowbeer]

@jcommaret if unspecified, nodejs/v8 configures max-old-space based on the memory limit.

For example, with a 2g memory limit, max old space is set to 1g by default.

The nodejs docs state that you might want to adjust this. For example, if you're only running one process in 2g memory then you might want to set max old space to 1536 in order to use more of the available memory.

For example, below is node:20 running with a memory limit of 2g and max old space of 1536mb:

docker run -it -m 2g \
  -e NODE_OPTIONS="--max-old-space-size=1536" \
  node:20 node -e 'console.log(v8.getHeapStatistics().heap_size_limit/(1024*1024))'

However, it does not make sense for you to set max old space (or create an issue) unless you know what the memory limit is.

[jcommaret]

ok, let's remove the --max-old-space flag.

[jcommaret]

I removed the --max-old-space-size to allow nodejs use the space he wants - still crash with 6134 mo (the maximum i can allocate to my virtual machine, not using --max-old--space-flag )

Mac OS is working with 2048 mo.

Don't understand why there is such a difference.

Let's try on Linux x86