Simplify vm.Module

[GeoffreyBooth]

One thing that comes up in user feedback about the vm.Module API is that it requires excessive boilerplate, as you can see in the example in the docs: https://nodejs.org/api/vm.html#class-vmmodule. I think that most users want a straightforward ESM equivalent to vm.Script, without needing to define a context or a custom function for linking; the default should be to link modules as Node itself normally does. See also #43288 (comment).

[legendecas]

There are several open issues on vm.SourceTextModule, like:

• SourceTextModule: Using dynamic imports while passing cachedData throws TypeError: Invalid host defined options #43663
• await Promise.resolve("import('./mod.mjs')").then(eval) rejects/segfaults in REPL #43681

I'm still looking into these problems as part of the effort on bringing module support to ShadowRealm in Node.js, but they depend on the work on the v8 side to enable arbitrary values to be saved in the host_defined_options and the v8 team is also working on refactoring how host_defined_option takes effects on code caching (https://docs.google.com/document/d/1vyfTs9pJVeDeguSVhYts3uolkhPk03eg5IvJeSR4IKI/edit#).

Also, there is a roadmap issue for stabling VM modules: #37648.

[devsnek]

closing as a duplicate of #37648

[GeoffreyBooth]

Thanks. I guess what's missing in that issue that I have here are the UX concerns, about using good defaults so that users can reduce boilerplate. I couldn't find an issue for that part, which maybe we could tackle now while we wait for V8?

[devsnek]

there are a lot of footguns you can hit if you are not careful with the vm api. for example when you point vm modules at node's main context, you create a memory leak which can't be fixed without changes to v8. i'm all for improving the vm api of course, but be wary that you'll likely hit lots of annoying roadblocks on the v8 side.

[GeoffreyBooth]

I didn't say it would be easy 😀 but I think reducing the complexity is a worthy goal. If that request isn't captured elsewhere, perhaps this issue should stay open to track that?

[GeoffreyBooth]

Reopening to focus on the UX changes. If that’s duplicated elsewhere we can close.

[axkibe]

One API minor change that would simplify code a lot in I guess many use cases is to allow linker function in case the context is current execution to return a "Module Namespace Object" just like importModuleDynamically does already. But the linker doesn't like that.

That means, in case one does not want to handle the linking with a custom function but forward it to node, one could simply "return import( specifier );" (also I guess a common case, user wants to extend something new, but for all other case have node.js handle it)

Right now as far I understood, one must encapsulate a syntheticModule around it as like so:

    const mod = await import( specifier);
    const exportNames = Object.keys( mod );
    const imported =
        new vm.SyntheticModule(
            exportNames,
            ( ) =>{ exportNames.forEach( key => imported.setExport( key, mod[ key ] ) ); },
            {  identifier: specifier }
        );
    return imported;

would become

    return import( specifier );

I believe even no await needed, may return the promise directly. Actually the first case misses caching the results in a Map, something that also would be needed.

PS: I currently finished changing a more complex system from CJS vm.Script.runInThisContext() with simulating handing down module/require to ESM with the new experimental SourceTextModule API, and while I agree there is room for improvement I must say, in practice everything is already so much cleaner now, because things are way more structured. Overall a huge improvement, albeit more boilerplate needed.