Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support preallocated pthread & wasm worker from blob #22565

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ifiddynine
Copy link
Contributor

@ifiddynine ifiddynine commented Sep 13, 2024

This amends existing mainScriptUrlOrBlob logic to make sure that we can preload JS for pthread and wasm workers reliably.

On the pthread side, this amends PTHREAD_POOL_SIZE warmup logic to allow waiting for a promise on top of existing logic in PThread.allocateUnusedWorker. Otherwise, there would be a race which would probably mean the JS file would be loaded several times anyways.

On the wasm worker side, simply fixes the ability to provide a blob - previously it would only accept a javascript string. We use WASM_WORKERS=2 to keep this working, mixing wasm workers + pthread.

See below for example of how it's used. I considered just refactoring the scriptUrlOrBlobPromise logic to automatically cache all worker startups, but figured that would be harder to get integrated since it would break backwards compatibility & tests that use it in emscripten main.

Module['mainScriptUrlOrBlobPromise'] = fetch('myscript.js', { priority: 'high' })
    .then(response => response.text())
    .then(workerScript => {
    Module['mainScriptUrlOrBlob'] = new Blob([workerScript], { type: 'application/javascript' });
});

} else
#endif
{
preallocateWorkers();
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about instead updating init() and leaving initMainThread() unchanged.

e.g.:

#if expectToReceiveOnModule('mainScriptUrlOrBlobPromise')
if (Module['mainScriptUrlOrBlobPromise']) Module['mainScriptUrlOrBlobPromise'].then(PThread.initMainThread());
else
#endif
PThread.initMainThread()

This would avoid the new/extra closure.

@@ -182,7 +182,7 @@ if (ENVIRONMENT_IS_WASM_WORKER
'mem': wasmMemory,
#else
'wasm': wasmModule,
'js': Module['mainScriptUrlOrBlob'] || _scriptName,
'js': URL.createObjectURL(Module["mainScriptUrlOrBlob"]) || _scriptName,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is fix for the case when mainScriptUrlOrBlob is a blob? Is this because blobs cannot be serialized but object URLs can?

Can we add a test for this? And perhaps make it a separate PR and it seems somewhat orthogonal.

PThread.allocateUnusedWorker();
}
};
#if expectToReceiveOnModule('mainScriptUrlOrBlobPromise')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is mainScriptUrlOrBlobPromise? I don't see that anywhere in the existing source code? Is the idea that someone could add this to -sINCOMING_JS_MODULE_API? If so I wonder if we should document it somewhere.

If that creation of the main thread needs to be delayed until the mainScriptUrlOrBlob is ready can't this be done via existing mechanisms? For example, any of these approaches:

  1. Using MODULARIZE and delaying the instantiation of the module.
  2. Delaying the import of the emscripten-generated script
  3. Using addRunDependency ? (this one might not work since I'm not sure it blocks initMainThread.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants