Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bzlmod sysroot example test not working #234

Closed
steve-261370 opened this issue Oct 22, 2023 · 6 comments · Fixed by #235
Closed

bzlmod sysroot example test not working #234

steve-261370 opened this issue Oct 22, 2023 · 6 comments · Fixed by #235

Comments

@steve-261370
Copy link

I cloned the repo and went into the tests directory.

The following command, which uses the llvm sysroot toolchain, with bzlmod disabled, works:

bazel test --enable_bzlmod=false --extra_toolchains=@llvm_toolchain_with_sysroot//:cc-toolchain-x86_64-linux //:all

But when bzlmod is enabled, the build fails

bazel test --enable_bzlmod=true --extra_toolchains=@llvm_toolchain_with_sysroot//:cc-toolchain-x86_64-linux //:all

The error returned is:

ERROR: An error occurred during the fetch of repository 'toolchains_llvm~override~llvm~llvm_toolchain_with_sysroot': Traceback (most recent call last): File "/home/steve-261370/.cache/bazel/_bazel_steve-261370/fe79d53e53f1045ef52b7f75d9f074d8/external/toolchains_llvm~override/toolchain/internal/configure.bzl", line 87, column 57, in llvm_config_impl llvm_dist_path_prefix = _pkg_path_from_label(llvm_dist_label) File "/home/steve-261370/.cache/bazel/_bazel_steve-261370/fe79d53e53f1045ef52b7f75d9f074d8/external/toolchains_llvm~override/toolchain/internal/common.bzl", line 144, column 13, in pkg_path_from_label if label.workspace_root: Error: 'workspace_root' is not allowed on invalid Label @[unknown repo 'llvm_toolchain_llvm' requested from @toolchains_llvm~override]//:BUILD.bazel

It looks like the github tests only run through the sysroot tests with bzlmod=false. I am running bazel 6.4.0 on a debian-based system.

@patrice332
Copy link

Hitting the same issue

@fmeum
Copy link
Member

fmeum commented Jan 11, 2024

We are also hitting this. For now, you can work around this by specifying a canonical label:

llvm = use_extension("@toolchains_llvm//toolchain/extensions:llvm.bzl", "llvm")
llvm.toolchain(
    llvm_version = "16.0.4",
    sysroot = {
        # The extension doesn't handle labels to external repositories correctly: It does not map
        # apparent names such as @org_chromium_sysroot_linux_x64 to the correct Bazel-internal
        # canonical repository name. As a workaround, specify the canonical name directly.
        # TODO: Get rid on this dependency of Bazel implementation details when toolchains_llvm
        #       offers an attribute of type label rather than string.
        "linux-x86_64": "@@_main~_repo_rules~org_chromium_sysroot_linux_x64//:sysroot",
    },
)
use_repo(llvm, "llvm_toolchain")

@mering
Copy link
Contributor

mering commented Jan 11, 2024

How can I find out the canonical label of a transitive non-module dependency (made visible via use_repo())?

@fmeum
Copy link
Member

fmeum commented Jan 11, 2024

You can bazel query --canonical_labels for it using a regular label.

@mering
Copy link
Contributor

mering commented Jan 15, 2024

You can bazel query --canonical_labels for it using a regular label.

Thanks for the hint. I got it working by using --consistent_labels.

@Lev1ty
Copy link

Lev1ty commented Mar 11, 2024

the full command bazel query --consistent_labels @org_chromium_sysroot_linux_x64//:sysroot

the below snippet worked well for me

bazel_dep(name = "toolchains_llvm", version = "0.10.3")

http_archive = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
  name = "org_chromium_sysroot_linux_x64",
  integrity = "sha256-G+YOfEVqvFkKYTxk+rTqx2MsgexvInNKYbU2aaRAc0Y=",
  urls = ["https://commondatastorage.googleapis.com/chrome-linux-sysroot/toolchain/2028cdaf24259d23adcff95393b8cc4f0eef714b/debian_bullseye_amd64_sysroot.tar.xz"],
  build_file_content = """
filegroup(
  name = "sysroot",
  srcs = glob(["*/**"]),
  visibility = ["//visibility:public"],
)
""",
)

llvm = use_extension("@toolchains_llvm//toolchain/extensions:llvm.bzl", "llvm")
LLVM_VERSION = "16.0.4"
llvm.toolchain(llvm_version = LLVM_VERSION)
use_repo(llvm, "llvm_toolchain", "llvm_toolchain_llvm")
llvm.toolchain(
  name = "llvm_toolchain_with_sysroot",
  llvm_version = LLVM_VERSION,
  # https://github.com/bazel-contrib/toolchains_llvm/issues/234
  # aspect query --consistent_labels @org_chromium_sysroot_linux_x64//:sysroot
  sysroot = {"linux-x86_64": "@@_main~_repo_rules~org_chromium_sysroot_linux_x64//:sysroot"},
  toolchain_roots = {"": "@@toolchains_llvm~~llvm~llvm_toolchain_llvm//"},
)
use_repo(llvm, "llvm_toolchain_with_sysroot")
register_toolchains("@llvm_toolchain//:all")

siddharthab added a commit that referenced this issue Mar 13, 2024
#235)

Currently the repo rule and tag class accept string-form labels for
toolchain root packages and sysroots. Under `bzlmod` this is problematic
because users may pass us labels that point at repos that are not in
this module's repo mapping. To support such labels, they need to be
passed to us as actual `Label`s (not strings).

This necessitates some (**breaking**) changes to interface for the
module extension tags. The repo rule interface remains the same.

For the "llvm" module extension, two new tags have been introduced:
- `toolchain_root`, and 
- `sysroot` 

Alternatives considered:
1. Using a `label_keyed_string_dict` would not work if we still want to
    support absolute paths in these attributes.
2. Using string aliases instead of string labels, and then a separate
    attribute for a side table that maps labels to their aliases could also
    work. This would have to be done only for the module extension
    and not the repo rule, because specifying labels in repo rules
    eagerly fetches them.

I've also enabled bzlmod-enabled tests for the system paths, absolute
paths, and cross tests in CI.

Fixes #234. cc: @steve-261370

---------

Co-authored-by: Siddhartha Bagaria <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants