SCons: Add an option to detect C++ modules recursively #43057
Conversation
This adds `custom_modules_recursive` which allows to detect and collect all nested C++ modules which may reside in any directory specified by `custom_modules` option. The detection logic is made to be more strict because `SCSub` may be used for organizing hierarchical builds within a module itself, so the existence of `register_types.h` and `config.py` is checked as well (these are all required for a C++ module to be compiled by Godot). For performance reasons, built-in modules are not checked recursively, and there's no benefit of doing so in the first place. It's now possible to specify a directory path pointing to a *single* module, as it may contain nested modules which are detected recursively.
Calinou
added
enhancement
topic:buildsystem
cherrypick:3.x
| if is_engine(child): | ||
| continue | ||
| to_search.insert(0, child) | ||
| return modules | ||
|
|
||
|
|
||
| def is_module(path): |
There was a problem hiding this comment.
This method can be moved to detect_modules method above as well, but technically it would break compat for 3.2.
|
Thanks! |
|
Cherry-picked for 3.2.4. |
akien-mga
removed
the
cherrypick:3.x
|
This seems to break Python 2 support in the |
|
I think it would be enough to just disable recursive detection of modules in Python 2, I'm not sure if there's a better way to resolve this issue... EDIT: Nevermind, it's a syntax error, I think it cannot be worked around. I'll try to fix this. |
|
I think we can just move those nested functions to the outer scope and it should work, even if it's less elegant? Maybe with a leading underscore or prefix to show they're not part of the public API. For |
That's not a very pretty way to parse a string from a python file though, there should definitely be cleaner and more reliable ways to do this without using `exec()`.
|
As mentioned in #45897, this PR doesn't seem to work for me on Linux with Python 3.8.7 and SCons 4.1.0: |
|
I've tested this on SCons 4.0.1, so likely caused by SCons upgrade, strange error. Python 3.9.1. |
|
Same issue with SCons 4.0.1 for me. The problem seems to be around that same I have a lot of The current code seems to choke on some of them. |
|
I suspect that this may be something to do with reliability of engine detection. What is your current working directory? It may detect the same "version.py" and the same "short_name", but does not necessarily mean that's from Godot. 🙂 I guess EDIT: but seeing your structure, this also may not be enough. I'd probably just document that "custom_modules" should point to actual modules, your current way of specifying |
`exec()` was not a good idea as it assumes a certain type of `version.py` file similar to Godot's own file, which is not always a reliable assumption (see godotengine#43057 (comment)). Also restores Python 2 support for the 3.2 branch.
`exec()` was not a good idea as it assumes a certain type of `version.py` file similar to Godot's own file, which is not always a reliable assumption (see #43057 (comment)). Also restores Python 2 support for the 3.2 branch. (cherry picked from commit 75910d1)
`exec()` was not a good idea as it assumes a certain type of `version.py` file similar to Godot's own file, which is not always a reliable assumption (see godotengine#43057 (comment)). Also restores Python 2 support for the 3.2 branch.
Closes godotengine/godot-proposals#1619.
This adds
custom_modules_recursiveSCons build option which allows to detect and collect all nested C++ modules which may reside in any directory specified bycustom_modulesoption. This option is enabled by default for external modules to achieve the greatest convenience for users.The detection logic is made to be more strict because
SCSubmay be used for organizing hierarchical builds within a module itself, so the existence ofregister_types.handconfig.pyis checked as well (these are all required for a C++ module to be compiled by Godot, likely throughout existence of Godot).For performance reasons, built-in modules are not checked recursively, and there's no benefit of doing so in the first place, so this has almost no performance impact for built-in modules.
Important improvement
It's now possible to specify a directory path pointing to a single module, and it may contain nested modules which are detected recursively. This is another improvement if you just want to compile a single module alone, as compiling with
custom_moduleswas cumbersome if you also have other modules in the same parent directory (which also may or not be compatible with current Godot source). Scripts can be further simplified by removing various module detection hacks, for example in Goost'sSConstruct:These changes are backward-compatible with Godot 3.2 (not trivially cherry-pickable).