cling based C++ as scripting language / hot code reload

[blockspacer]

cling based C++ as scripting language / hot code reload
Why? Able to run C++ script in runtime or compile it for max speed ( as in example https://github.com/derofim/cling-cmake )
Also: May be used to improve modules compile time

HOT code reload
possible approaches:

store app state
fix cling undo for files
https://root-forum.cern.ch/t/loading-unloading-class-as-interpreted-macro-in-cling-multiple-times/32976/2

execute cling code to change callbacks & variables
nested cling::Interpreter with multiple cling::MetaProcessor
IDK how to do it, but you can create child cling::Interpreter

UPD:
In theory, Cling may replace gdscript for apps that don`t provide gdscript modification to end users. Very useful if you are developing browser games (even plugins may be created with .dll or .so files, so Cling may be useful for any app), so you will be able to remove gdscript and cling from production compile step (to reduce app size and make it faster).

[ghost]

I just had this idea myself. I have no idea where to begin, but I'd like to be a contributor to this idea. Where might one begin with this?

[blockspacer]

@thirtyTwo-32

I just had this idea myself. I have no idea where to begin, but I'd like to be a contributor to this idea. Where might one begin with this?

Need to decide how you want to reload C++ module in editor UI (refresh button?), game UI (refresh button?), console (command?) or network (localhost/remote), ...

I propose to add UI window to godot editor where C++ modules can make changes (like add button to refresh module subsystem, create entry in top menu, etc).

Can C++ module extend godot editor that way? Any links will be usefull.

Note that it may be useful to reload C++ code while game is running, so need to provide API that can be used by game.

Module can use some cling utility library, that provides useful functions like below:

• createInterpreter, but with configurable parameters like https://github.com/derofim/cling-cmake/blob/master/src/main_cling.cpp#L364
• registerSources that just adds module sources (source file order is very important) to some map like https://github.com/derofim/cling-cmake/blob/master/src/main_cling.cpp#L454 or pass C++ variables into Cling  derofim/cling-cmake#4
• refreshSources that re-creates interpreter. May lose app state (developer can code app state saving, anyway, hot hot reload) or require editing to editor (if changed too much logic, for example - need to re-register gdscript bindings etc)
• interpretCode - just runs provided C++ code, once. For example, developer changes frequently only one function. That function can read C++ code in dev mode and run interpretCode by itself.
  See as example https://github.com/blockspacer/CXXCTP/blob/master/src/template_engine/CXTPL_AnyDict.cpp#L96

native C++ can provide complex data structures to cling interpreter via hex pointers
I use std::hex like so https://github.com/blockspacer/CXXCTP/blob/d71987214e8c17023cee9dd15086f9c7e632fbc9/include/template_engine/I_Dict.hpp#L30

cling can pass data to native C++ via cling::Value like so https://github.com/blockspacer/CXXCTP/blob/53f819bfcfff68f43e3568eedce11e405dfe20ee/src/clangPipeline.cpp#L406
than you can check cling::Value and get data like so

                    cling::Value result;
// ...
                    if(result.hasValue() && result.isValid()
                        && !result.isVoid()) {
                      void* resOptionVoid = result.getAs<void*>();
                      auto resOption =
                        static_cast<llvm::Optional<std::string>*>(resOptionVoid);
// ...
                        delete resOption; /// \note frees resOptionVoid memory
// ...

https://github.com/blockspacer/CXXCTP/blob/22b9469312e259131f9e77d64ca4a5cc8947b119/src/clangPipeline.cpp#L440

Note that cling::Value have no conversion to std::string (see above), but you can cast to void pointer that can store anything.

Cling can`t refresh function / includes in plug-and-play manner. Developer have some options:

1. Register almost all codebase in Cling. May be not good solution for performance-critical code, interpreter may slow things a lot.
2. Register big, but logically separated parts of codebase in one Cling interpreter. May require architecture changes (pass data to cling via std::hex pointers). May require to move all #includes into separate file to load them once. Functions may be defined as std::function, so you will be able to assign different lambdas by hand at any time (like ModuleA->setCallbackB(foo)). Class structure may be not modifiable, only callbacks/variables will be re-assignable.
3. (good approach, covers (1)) Register big, but logically separated parts of codebase in separate Cling interpreters (one interpreter for each subsystem - separated part of codebase). May require architecture changes (pass data to cling via std::hex pointers). Each module can`t share data with others (anyway - good OOP style), must use callbacks or signals or something other you prefer. For hot code reload (while app running) - may require to store app state somehow, anyway - application dependent, just provide save/load functionality. Performance critical code may be compiled as usual (very important for games).
4. (good approach, may be combined with (3)) Register functions from one source file. With source file reloading by demand and then calling required function.
5. (good approach, may be combined with (3)) Cling may be integrated with some ECS system (logic already separated from data)
6. Use unload. Feature is experimental. See try feature: support for redefinitions (DefinitionShadower) derofim/cling-cmake#5

tutorials or samples? see derofim/cling-cmake#8

In theory, Cling may replace gdscript for apps that don`t provide gdscript modification to end users. Very useful if you are developing browser games (even plugins may be created with .dll or .so files, so Cling may be useful for any app), so you will be able to remove gdscript and cling from production compile step (to reduce app size and make it faster).

[KoBeWi]

Feature and improvement proposals for the Godot Engine are now being discussed and reviewed in a dedicated Godot Improvement Proposals (GIP) (godotengine/godot-proposals) issue tracker. The GIP tracker has a detailed issue template designed so that proposals include all the relevant information to start a productive discussion and help the community assess the validity of the proposal for the engine.

The main (godotengine/godot) tracker is now solely dedicated to bug reports and Pull Requests, enabling contributors to have a better focus on bug fixing work. Therefore, we are now closing all older feature proposals on the main issue tracker.

If you are interested in this feature proposal, please open a new proposal on the GIP tracker following the given issue template (after checking that it doesn't exist already). Be sure to reference this closed issue if it includes any relevant discussion (which you are also encouraged to summarize in the new proposal). Thanks in advance!

(btw there's already a related proposal: godotengine/godot-proposals#573)