extra_targets, user configurable toolchains, and wasm support

[rrbutani]

(this PR builds on #75; that PR should be reviewed/merged before this one)

This PR is not ready for merging yet but I thought I'd open the PR anyways to get some feedback and see if this is a use case the maintainers are interested in.

---

This PR adds support for extra_toolchains, an attribute on configure that lets users register additional toolchains for different target triples:

load("@com_grail_bazel_toolchain//toolchain:rules.bzl", "llvm_toolchain")
llvm_toolchain(
    name = "llvm_toolchain",
    llvm_version = "8.0.0",
    extra_targets = [
      "wasm32-unknown-wasi",
    ],

    # Extra targets can have their sysroots overriden too:
    sysroot = {
        "linux": "@some_example_sysroot_repo//:linux_sysroot",
        "darwin": "@some_example_sysroot_repo//:macos_sysroot",

        "linux_wasm32-unknown-wasi": "@some_example_sysroot_repo//:wasi_sysroot",
        "darwin_wasm32-unknown-wasi": "@some_example_sysroot_repo//:wasi_sysroot",
    },
)

The toolchains registered with extra_targets have the appropriate constraint values so that Bazel will use them as necessary through toolchain resolution.

---

In extending the internals of this repo to support extra_targets, cc_toolchain_config and other internal macros were made a fair bit more general. This PR also tweaks these macros to be usable externally, allowing users to make small modifications to the configured toolchains without needing to modify this repo:

# WORKSPACE
# (parts to set up `@llvm_toolchain` have been elided; see above)

llvm_toolchain(
    name = "llvm_toolchain",
    llvm_version = "8.0.0",

    # NOTE: This is required to set up toolchains outside of `@llvm_toolchain`, unfortunately
    absolute_paths = True,
 )

# This registers the default toolchains.
load("@llvm_toolchain//:toolchains.bzl", "llvm_register_toolchains", "register_toolchain")

llvm_register_toolchains()

# Now let's make our own:
http_archive(
    name = "thumbv7-sysroot",
    urls = ["example.com"],
)
register_toolchain("//tests:custom_toolchain_example")

# BUILD file:
# Example Custom Toolchain:
load("@llvm_toolchain//:cc_toolchain_config.bzl", "cc_toolchain_config")

# Docs for this function and `overrides` are in `cc_toolchain_config.bzl.tpl`.
cc_toolchain_config(
    name = "custom_toolchain_example_config",
    host_platform = "linux",
    custom_target_triple = "thumbv7em-unknown-none-gnueabihf",
    overrides = {
        "target_system_name": "thumbv7em-unknown-none-gnueabihf",
        "target_cpu": "thumbv7em",
        "target_libc": "unknown",
        "abi_libc_version": "unknown",

        # If you omit this, be sure to depend on
        # `@llvm_toolchain:host_sysroot_components`.
        # "sysroot_path": "external/thumbv7-sysroot/sysroot",

        "extra_compile_flags": [
            "-mthumb",
            "-mcpu=cortex-m4",
            "-mfpu=fpv4-sp-d16",
            "-mfloat-abi=hard",
        ],
        "omit_hosted_linker_flags": True,
        "omit_cxx_stdlib_flag": False,
        "use_llvm_ar_instead_of_libtool_on_macos": True,
    }
)

load("@com_grail_bazel_toolchain//toolchain:rules.bzl", "conditional_cc_toolchain")
conditional_cc_toolchain(
    name = "custom_toolchain",
    toolchain_config = ":custom_toolchain_example_config",
    host_is_darwin = False,

    sysroot_label = "@llvm_toolchain//:host_sysroot_components", # use this if not overriding
    # sysroot_label = "@thumbv7-sysroot//:sysroot", # override

    absolute_paths = True, # this is required for toolchains set up outside of `@llvm_toolchain`, unfortunately
    llvm_repo_label_prefix = "@llvm_toolchain//",
)

# Constraints come from here: https://github.com/bazelbuild/platforms
toolchain(
    name = "custom_toolchain_example",
    exec_compatible_with = [
        "@platforms//cpu:x86_64",
        "@platforms//os:linux",
    ],
    target_compatible_with = [
        "@platforms//cpu:armv7", # `v7e-mf` has not yet made it to stable Bazel?
        # "@platforms//os:none",
    ],
    toolchain = ":custom_toolchain",
    toolchain_type = "@bazel_tools//tools/cpp:toolchain_type",
)

The interface is currently extremely rough and is just intended to be a way to prototype support for other targets without needing to modify this repo. If there's interest, there's lots that can be done to turn this into a user-friendly API.

---

Currently, only the wasm32-unknown-wasi target triple is tested and actually working. However this PR lays the foundation for support for other targets; it should just be a matter of adding sysroot and compiler_rt locations for other targets and fine-tuning the LLVM target triple to Bazel constraint value mappings.

---

As an aside, even in adding support for just a single additional (not-hosted) target the ways in which using unix_cc_toolchain_config.bzl (#75) limit us becomes apparent (we would like to have the toolchain for wasm targets not support the dynamic linking actions since they just error; instead we're forced to use --features=-supports_dynamic_linker on the command line or transitions/selects or other such workarounds).

So far this has been manageable but this definitely casts some doubt on whether #75 is the way to go.

Another thing to consider is whether upstream Bazel would be willing to make unix_cc_toolchain_config.bzl slightly more general/whether or not our use case aligns with its intended purpose.

[dzbarsky]

@rrbutani Any chance you are still interested in pursuing this? wasm support would be really nice

[rrbutani]

@dzbarsky I'm still interested but I think the approach I took was pretty general; I'm not sure if it aligns with the goals of this project.

Are you interested in just wasm support or also in the extra targets/configuration bits as well?

[dzbarsky]

I suspect a lot of users end up applying small patches to twiddle some of the options that you expose as extra configuration, so I'm not sure if its worth the complexity. For me the main benefit would be to have a way to target wasm using a platform_transition_filegroup or similar in usercode and have the cc toolchain resolution work out correctly; I'm not sure if that requires extra_targets.

[rrbutani]

For me the main benefit would be to have a way to target wasm using a platform_transition_filegroup or similar in usercode and have the cc toolchain resolution work out correctly; I'm not sure if that requires extra_targets.

I won't speak for the active maintainers of this project but I'd guess that unconditionally registering wasm/wasi toolchains (and fetching the wasi-sdk sysroot for the latter) probably isn't too controversial; there's probably no need to grow configuration options like extra_targets for this use case.

Unfortunately I won't have the cycles to work on something like this for at least the next few weeks; I'm happy to answer questions though.

[siddharthab]

My thinking was that existing users can go through https://github.com/grailbio/bazel-toolchain#supporting-new-target-platforms and submit PRs for their target platforms, including WASM. I am a little bit reluctant to also manage the mapping between LLVM versions and WASM SDK in this project, so maybe it can be left to the users to configure the right sysroot. Maybe just adding a test for WASM here can serve as an example.

[siddharthab]

If this is not an active topic anymore, should we close it?