-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add super-build for cross-compiling HANNK #6374
Conversation
Stopping buildbots because these changes are not tested by the buildbots. |
@steven-johnson raised a very valid concern, which is that super-builds are complex. Here is why I believe they're worth the pain setting up (I don't actually think this is very hard to use). Let me summarize the prior design:
Here are a few deficiencies with the prior implementation:
The brittleness I'm most concerned about in this design is the inability to pick the host toolchain that will be used. Setting something like But the rabbit hole of toolchain nonsense doesn't end with the compiler selection, sadly... people have many reasons to pass custom flags (to work around bugs, try a special compiler, etc.). So unless CMake suddenly develops features to support multiple active toolchains (the devs are very resistant to this), it's better for end users to have more loosely coupled projects. That's why I went for a "build twice with export + find_package" approach. On the other hand, this is kind of a pain for people actually working on the project in a cross-compiling setting since they're too de-coupled; the cross-build doesn't know how to call the host-build. That's what the super-build solves. The way I designed the super-build:
It's also opt-in, so if there's something wrong with the super-build, a user can still call the build twice manually to get un-stuck, using standard CMake features, without patching our build sources. Finally, the script has some knowledge about how to set up the super-build for common scenarios (e.g. Android and WASM) and also knows when a super-build isn't required. It is also opt-in. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, but going to see if Andrew or Dillon have any thoughts
The single failure is |
Doesn't look like custom_cuda_context. Looks like a deadlock?! https://buildbot.halide-lang.org/master/#/builders/181/builds/935 |
Whoops! Bad assumption... regardless, this PR has nothing to do with it. |
Buildbots are green! |
I guess no one else has opinions, so LGTM, let's land this |
After #6361 the cross-compiling workflow requires building HANNK twice: once for the host-run generators (with a host toolchain) and again with the cross toolchain. This PR adds an opt-in superbuild that orchestrates running both builds and correctly updates the world with a single call to Ninja.
The
configure_cmake.sh
script learned how to callget_host_target
to determine whether the givenHL_TARGET
requires cross compiling. When not cross-compiling, it invokes the build directly. When cross compiling, it invokes the super-build. It doesn't actually run the build (it's called "configure", after all) in either case. It checks a few basic sanity conditions on the env-vars it reads, now, too.The superbuild has a few special features to make using it more convenient:
HANNK_HOST_<VAR>
are forwarded to the host build as<VAR>
HANNK_CROSS_<VAR>
are forwarded to the cross build as<VAR>
cmake --install
andctest
work from the top-level build directory.