Proposal: Copy GDExtension DLLs when starting the editor on Windows so the original DLL isn't locked and can be rebuilt

[BastiaanOlij]

The problem we have today is that once the editor is open, it loads our dynamic library and on many platforms it locks this. The result is that you can't recompile the library to test changes while the editor is open.

In Godot 3 we had a reload feature however this is no longer a possible solution in Godot 4 as unloading an external will remove our classdb information potentially damaging the scenes and script currently loaded which could result in the user loosing work.

A potential alternative discussed during one of our GDExternal review meetings was to add some code when loading the externals that if we're in the editor, we first make a copy of the DLL into our .import folder and start this. This allows for the original DLL to be overwritten and the user being able to test changes in runtime.

With this solution the editor won't pick up added classes, properties or methods, that will still require a restart, but it will greatly reduce the need for restarting the editor in many situations.

[Zylann]

What about dependencies of the library?

With this solution the editor won't pick up added classes, properties or methods

So you mean the editor will still not pick up new changes, but the game will? (since the game restarts every time when pressing Play)

[BastiaanOlij]

What about dependencies of the library?

We still don't have any sort of dependency management but I suspect that will have similar issues if we unload a library.
I don't know the exact mechanism that Godot uses when reloading a GDScript file as it does have similar issues and deals with it. I must admit that the issues I've had in the past with unloading libraries may have been fixed, though especially libraries that implement system classes and singletons can have rather nasty consequences when reloaded which is why we added flags in Godot 3 to prevent reloading.

So you mean the editor will still not pick up new changes, but the game will? (since the game restarts every time when pressing Play)

Yes, that's the idea here. It's not perfect but a whole lot better then where we are today.

[Faless]

it loads our dynamic library and on many platforms it locks this.

As far as I know this is a windows-specific issue with file system access, so we should keep whatever hack we make windows-specific unless someone can actually reproduce the "permission denied" issue on other platforms.

[dsnopek]

Discussed at the GDExtension meeting, and we think that a solution like this to work around the Windows DLL locking issue makes sense, but it should probably be opt in, so that it can be used by developers working on a GDExtension but not by users.

There is some concern that this might not work for all GDExtensions. If that's the case this should be opt in, so it doesn't break anything for users. However, if that's not the case and this will always work, then this could be opt out.

This needs to also take into consideration the GDExtension's dependencies and copy those as well.

@vnen may take a look at it!