Add a small, baseline WebAssembly interpreter to Godot

[basicer]

Describe the project you are working on

Godot

Describe the problem or limitation you are having in your project

The current solution to using code written in other languages, GDExtension, comes with a few drawbacks. As soon as you add the first GDExtension to your project you take on additional complexity:

• Needing to maintain a system to create builds for every system -or- if using an Addon provided by the library, do they provide builds for all the platforms you want.
• Dealing with cross platform incompatibility when trying to adopt a third-party C++ library.
• “single.exe” distribution no longer works.
• Additional testing burden on each platform to make sure everything works.
• If using prebuilt binaries, trusting they haven't been tampered with.
• Unclear console support.

If your objective is to just reuse a small C library to do something like ed25519 validation, all that added complexity starts to make it not worth it.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

I propose we keep things simple, and just add a small, simple WASM pure-c interpreter such as Wasm3 with an API similar to https://github.com/ashtonmeuser/godot-wasm

A C++ library can be compiled to WebAssembly and accessed from GDScript though the common WebAassembly interface with minimal fuss. You can compile it once, test it on one platform, and be confident it will also work properly on all platforms, even ones you don't have access to. Likewise an asset built this way placed on the Asset Library will work for everyone to use with no additional work by the author.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

There have been quite a few proposals in this area: #147 , #5010 , #3370 , #9177 , and to some degree #10071 . This proposal builds on those attempting to find a goldilocks solution that maximizes the benefits of those proposal while minimizing the costs.

Some discussions on the tradeoffs of this proposal vs other proposals:

Why not use a faster WebAssembly engine such as Wasmer?
For many uses, the speed granted by including a JITing WebAssembly engine isn't worth the added complexity. Some platforms enforce W^X and won't ever work well with a JIT. Infact wone of the strengths of this approach if projects that need the speed can easily bolt on a different WASM runtime right before launch and not need to change their code, or ship different runtimes per platform.

But isn't an interpreter too slow?
Not really. I ran a quick test using the Sieve (999999) test from godot-wasm.
GDScript: 350ms
Wasmer: 5ms
Wasm3: 39ms

Does this bloat the engine?
No. On my machine, a prototype implementation increased the editor binary by about 150k.

Does this help Modding Support / Sandboxing
Yes! You could use this to either support your game being modable by loading wasm files and giving out a C style modding interface, or you could load an interpreter like lua inside the sandbox and pass scripts into it. In any case wasm code can only call functions explicitly registered with it, so it adds a nice layer of safety to users loading mods.

What about WASI?
It unclear to me the best way to implement WASI at this time, and its trivial to implement in GDScript for games that need it.

Could you load a WASM GDExtension this way?
Maybe? There are some challenges here because of pointer size and address space differences. I think its possible with a translation layer that could be prototyped in GDScript

Why WebAssembly over riscv or some other compilation target
The biggest advantage of WebAssembly is the ecosystem. The baseline wasm interpreter can be swapped out for a JIT one on a platform down the line if that performance is needed with no game code changes. Some existing libraries compiled to WASM can be used off the shelf with out recompiling. The tooling to build, validate, and test WASM blobs meant for embedding is much more developed than the alternative.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No

Is there a reason why this should be core and not an add-on in the asset library?

As discussed above, being core to Godot is a large part of the benefit.

[octanejohn]

i think wamr is better than wasm3 and smaller by selecting diffrent modes intepreted/jit/compiled
https://github.com/bytecodealliance/wasm-micro-runtime

[fire]

Since there are several implementations of VM interpreters in WASM, and including our implementation in RISC-V (https://github.com/libriscv/godot-sandbox), it's unclear which ones should be considered "official".

Typically, where we don't have a definitive answer, the Godot Engine team tends to leave it to GDExtension.

[basicer]

Since there are several implementations of VM interpreters in WASM, and including our implementation in RISC-V (https://github.com/libriscv/godot-sandbox), it's unclear which ones should be considered "official".

Typically, where we don't have a definitive answer, the Godot Engine team tends to leave it to GDExtension.

While this is a good approach in general, the problem here is main thing this proposal seeks to do is to provide an alternative to GDExtension for extensions that don't need all the added complexity that comes with maintaining GDExtensions and native code. It's not super important which implementation is chosen, as long as one makes its way into core.

[fire]

Since you are working on a wasm implementation, I can send you our notes from wasgo. https://github.com/V-Sekai/wasgo We abandoned wasgo a while back.

Is there better way to keep Callables alive than to store a linked list of them?

The idea we came up with was an integer index into a table of Callables that are replaced in the guest api.

The godot code has to reach directly into m3 to work around some methods missing from it's public API, consider adding public apis in wasm3 for these and upstream that.

It is unclear how responsive upstream wasm implementations can be.

Add UnitTests / Integration Tests

I recommend using godot engine's doc test. https://docs.godotengine.org/en/stable/contributing/development/core_and_modules/unit_testing.html

Better Examples

We released a wasgo wasm demo in 2021, maybe there's parts you can salvage https://github.com/V-Sekai/wasgo/releases/tag/v0.0.1

The demo is a plane responding to keyboard input and mouse input.

[basicer]

Since you are working on a wasm implementation, I can send you our notes from wasgo. https://github.com/V-Sekai/wasgo We abandoned wasgo a while back.

Very cool. Why did you abandon it? The risc-v implementation is undoubtedly super neat (I'm a huge risc-v fan!), but to me, WebAssembly seemed like it would be the less controversial choice for something in core. I'm not attached to wasm, more I'm interested in having a system that mitigates the cross-platform difficulties of GDExtension when the full power of native code isn't needed.

Do you think the risc-v proposal has a good chance of making it in?

[fire]

@basicer Personally if we keep godot-sandbox as a gdextension we don't have to argue for it, if enough people like it'll naturally be requested to be added to core. With my limited efforts I want to dedicate effort to my csg (mesh boolean) refactor.

https://gonzerelli.itch.io/demo

Here's a demo of godot-sandbox. We created a script that's written in c++ and then loaded into a riscv virtual machine. This is in wasm running riscv64 linux elf binary.

Here's the source code of the demo https://github.com/libriscv/godot-sandbox-demo

We're probably going to use godot-sandbox in our game.

[fire]

Why did you abandon it?

Code generating the entire surface area of godot engine api was too much. We started smaller and exposed apis because it means we can reason about it's sandbox properties.

Binding the entire Variant api with Dictionary and Array being references and the rest being copies was frustrating.