Release and reacquire memory

[penzn]

Originally posted by @photopea in #1300 (comment)

My visitors spend hours on my website without reloading it.

My wasm program runs once every 5 minutes, for 0.5 seconds each time. It is called by a synchronous JS function, which expects the output right after the wasm finishes. During the execution, the wasm program needs 700 MB of RAM.

I would like to release these 700 MB when the wasm program does not run. What JS object should I release (remove references to)? Next time my JS function runs, I want to run the wasm program again (in a synchronous way).

I was thinking about recompiling WASM each time my JS function runs, but all current browsers compine WASM asynchronously. Is there anything else I can do?

These lines do nothing:

wasm.instance.exports.memory.buffer = new ArrayBuffer(10);
wasm.instance.exports.memory = new WebAssembly.Memory({initial:10});

[penzn]

I could not find any discussion on releasing and re-acquiring memory, hence a new issue.

[tlively]

Disposing of the Instance object and all references to its exports or other properties should free it up to be garbage collected, including its memory. If you keep the Module object around, you will be able to cheaply instantiate it with a fresh memory every time your program needs to run without recompiling anything.

[photopea]

@tlively could you provide a sample of code? Or maybe a demo on JSFiddle?

[MaxGraey]

async function compile() {
  const response = await fetch('module.wasm');
  const buffer = await response.arrayBuffer();
  const module = await WebAssembly.compile(buffer);
  return module;
}

function getInstanceSync(module, importObject = {}) {
  return new WebAssembly.Instance(module, importObject);
}

function getInstanceAsync(module, importObject = {}) {
  return WebAssembly.instantiate(module, importObject);
}

const mod = await compile();

const instance1 = getInstanceSync(mod);
const instance2 = getInstanceSync(mod);
const instanceN = getInstanceSync(mod);

[photopea]

@MaxGraey It works well in Firefox, but in Chrome, I am getting a following error in the latest Google Chrome:

Uncaught (in promise) RangeError: WebAssembly.Instance is disallowed on the main thread, 
if the buffer size is larger than 4KB. Use WebAssembly.instantiate.

It has been mentioned before - Chrome does not allow synchronous compilation of large WASM for some reason. Is here any Chromium developer? Could you fix it? I think if someone decides to do it on the main thread, they already know what they are doing.

My app www.Photopea.com often uses 4 - 8 GB of RAM, it often blocks the main thread for 5 - 10 seconds or more (the user sits and waits), so I don't think compiling my 30 kB WASM on the main thread would ruin the user experience.

[dtig]

@MaxGraey It works well in Firefox, but in Chrome, I am getting a following error in the latest Google Chrome:

Uncaught (in promise) RangeError: WebAssembly.Instance is disallowed on the main thread, 
if the buffer size is larger than 4KB. Use WebAssembly.instantiate.

It has been mentioned before - Chrome does not allow synchronous compilation of large WASM for some reason. Is here any Chromium developer? Could you fix it? I think if someone decides to do it on the main thread, they already know what they are doing.

My app www.Photopea.com often uses 4 - 8 GB of RAM, it often blocks the main thread for 5 - 10 seconds or more (the user sits and waits), so I don't think compiling my 30 kB WASM on the main thread would ruin the user experience.

The synchronous compilation of large modules is discouraged exactly for the reason you mentioned above - i.e. it leads to a pretty bad user experience, and this limit is unlikely to be changed. If you have Chrome specific concerns, please follow up on the Chromium issue tracker.

[jerch]

@dtig

The synchronous compilation of large modules is discouraged ...

But why falls the Instance constructor under this in chrome? Isn't creating an instance from an existing module supposed to be cheap, as compilation should be done during module creation?
Note that I dont follow your reasoning, if you take your own argument serious, then chrome should forbid/cancel any sync JS code running longer than XY ms. Furthermore not allowing new Instance is not in line with the wasm interface spec draft, as it does not tell anything about "may throw on bigger modules".

[Jamesernator]

Note that I dont follow your reasoning, if you take your own argument serious, then chrome should forbid/cancel any sync JS code running longer than XY ms.

There's a few reasons this isn't done, one is that it would break existing sites. One of the leading philosophies of browser design is as much as possible to avoid breaking sites unneccessarily.

Another though is just that interrupting JS execution by itself is not safe, the reason .terminate() is safe on a worker is because the engine instance is torn down and discarded. Modern engines do a lot of code generation on the fly, interrupting execution of JS code could potentially interrupt at an unsafe point in code generation leaving attack vectors for things like arbitrary code execution and the like. This might be technically possible to resolve, but it would likely lead to a slowdown, and even then it would be a large change to most engines.

If JS were recreated today, blocking for long periods probably wouldn't be allowed on the main thread. But unfortunately JS is not a new language, it has to support all of the old stuff in addition to new stuff so it's not super viable to just make large changes like that.

But why falls the Instance constructor under this in chrome? Isn't creating an instance from an existing module supposed to be cheap, as compilation should be done during module creation?

Although I do agree that restricting Instance seems unnecessary. From my experience instantiation is very fast in all WASM I have dealt with, so it's unclear to me as to what kinds of modules are anticipated that would actually produce a slowdown on instantiation.

[rossberg]

The restrictions on the Instance (and even the Module) constructor were rather controversial, even among Chromium devs. The web platform team insisted, while others thought there is no coherent reason to do so, like pointed out here as well. But as @dtig mentioned, the Chromium tracker is the right place to lobby for lifting the restriction.

[andyb1979]

Hi everyone, wonder if you found a solution to this?

I have an issue where new/delete of large arrays is causing memory fragmentation due to limitations in the wasm memory model (see #1397 )

we need a way to collect all and allow application memory to shrink when done with long running calculations

[dtig]

There's a few different issues mentioned here, and depending on what you're doing currently the solution may be different. If the calculations are across instances, @tlively's suggested method of disposing instances seems like a good approach. For discarding application memory, please follow this comment in the issue you linked for a memory.discard discussion + prototype. The Chrome limitation in synchronous compile is raised to 8KB, but it would still be for user experience to block the main thread for large module compilation.

[andyb1979]

Disposing instances (see comment here ) seems like it could work

but what’s involved? Drop every reference to the module? Still trying to figure out how this works. Thanks!

[jerch]

Drop every reference to the module?

Not to the module, but all refs to the instance. If you also drop all refs to the module, you'll have to go through the whole bootstrapping again. By keeping the module around, you can get much faster to a new instance.

[andyb1979]

Thank you that’s V. helpful

[andyb1979]

Can confirm. Once all instances of wasmInstance are dropped and the instance is GC'd by JavaScript, the wasm heap is also deleted and memory returned to the host OS