Question: How to find a JDK runtime with `master`? #64

jpsim · 2022-07-21T15:07:21Z

I've been using rules_java 5.0.0 successfully, but I'll soon need to update to include recent changes in master to support the latest bazel 6.x rolling releases.

Prior to 338c418 the JDK runtime toolchain was being found correctly, but after that commit I get the following error:

ERROR: /.../BUILD:21:19: While resolving toolchains for target //{my-target}: no matching toolchains found for types @bazel_tools//tools/jdk:runtime_toolchain_type

Here's the complete error

INFO: Repository remotejdk11_macos_aarch64 instantiated at:
  /Users/jsimard/src/envoy-mobile/WORKSPACE:58:26: in <toplevel>
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/envoy_mobile/bazel/envoy_mobile_dependencies.bzl:53:24: in envoy_mobile_dependencies
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/envoy_mobile/bazel/envoy_mobile_dependencies.bzl:62:28: in kotlin_dependencies
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/rules_java/java/repositories.bzl:469:23: in rules_java_dependencies
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/rules_java/java/repositories.bzl:233:10: in remote_jdk11_repos
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/bazel_tools/tools/build_defs/repo/utils.bzl:233:18: in maybe
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/rules_java/toolchains/remote_java_repository.bzl:48:17: in remote_java_repository
Repository rule http_archive defined at:
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/bazel_tools/tools/build_defs/repo/http.bzl:353:31: in <toplevel>
ERROR: /Users/jsimard/src/envoy-mobile/library/kotlin/io/envoyproxy/envoymobile/BUILD:21:19: While resolving toolchains for target //library/kotlin/io/envoyproxy/envoymobile:envoy_lib_kt: no matching toolchains found for types @bazel_tools//tools/jdk:runtime_toolchain_type

I'm probably not specifying something I should be, or perhaps this is considered a bug? I'm not sure, so I'm phrasing this as a question.

To reproduce you can clone Envoy Mobile and run the following command:

./bazelw build android_dist --config=android --override_repository=rules_java=/path/to/rules_java

I'll work to reduce and simplify the repro case but I wanted to share now in case the solution was obvious to more experienced users or maintainers of rules_java.

Thanks!

The text was updated successfully, but these errors were encountered:

This reverts commit 338c418. This is probably not the right fix, but it fixes bazelbuild#64 for me.

comius · 2022-07-21T17:57:01Z

I can work with a large reproducer.

Reverting would cause other problems. There's a usecase where JDK gets packaged for deployment, so it needs to be matched by the target.

Is this only problem for android?

You can get extra debug by adding flag:
--toolchain_resolution_debug='@bazel_tools//tools/jdk:runtime_toolchain_type'

jpsim · 2022-07-21T18:36:43Z

Is this only problem for android?

Yes, but only in the sense that we don't compile Java for any other platform, we compile C/C++/Python/ObjC/Swift for other platforms that aren't affected by this, but maybe that's obvious.

Actually it looks like this envoyproxy/envoy#22356 fixes the issue, but now bazel can't find the C++ toolchain:

ERROR: /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/bazel_tools/tools/cpp/BUILD:60:19: While resolving toolchains for target @bazel_tools//tools/cpp:current_cc_toolchain: No matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type. Maybe --incompatible_use_cc_configure_from_rules_cc has been flipped and there is no default C++ toolchain added in the WORKSPACE file? See https://github.com/bazelbuild/bazel/issues/10134 for details and migration instructions.

Maybe rules_java needs to use use_cpp_toolchain() somewhere?

Update:

Full error

$ ./bazelw build android_dist --config=android --override_repository=rules_java=/Users/jsimard/src/rules_java                                                                                                                            
INFO: Build option --toolchain_resolution_debug has changed, discarding analysis cache.
INFO: Repository remotejdk11_macos_aarch64 instantiated at:
  /Users/jsimard/src/envoy-mobile/WORKSPACE:58:26: in <toplevel>
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/envoy_mobile/bazel/envoy_mobile_dependencies.bzl:52:24: in envoy_mobile_dependencies
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/envoy_mobile/bazel/envoy_mobile_dependencies.bzl:61:28: in kotlin_dependencies
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/rules_java/java/repositories.bzl:517:23: in rules_java_dependencies
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/rules_java/java/repositories.bzl:251:10: in remote_jdk11_repos
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/bazel_tools/tools/build_defs/repo/utils.bzl:233:18: in maybe
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/rules_java/toolchains/remote_java_repository.bzl:48:17: in remote_java_repository
Repository rule http_archive defined at:
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/bazel_tools/tools/build_defs/repo/http.bzl:353:31: in <toplevel>
INFO: Repository maven instantiated at:
  /Users/jsimard/src/envoy-mobile/WORKSPACE:58:26: in <toplevel>
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/envoy_mobile/bazel/envoy_mobile_dependencies.bzl:52:24: in envoy_mobile_dependencies
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/envoy_mobile/bazel/envoy_mobile_dependencies.bzl:62:18: in kotlin_dependencies
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/rules_jvm_external/defs.bzl:121:19: in maven_install
Repository rule coursier_fetch defined at:
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/rules_jvm_external/coursier.bzl:1173:33: in <toplevel>
INFO: Repository android_tools instantiated at:
  /DEFAULT.WORKSPACE.SUFFIX:413:13: in <toplevel>
Repository rule http_archive defined at:
  /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/bazel_tools/tools/build_defs/repo/http.bzl:353:31: in <toplevel>
ERROR: /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/bazel_tools/tools/cpp/BUILD:60:19: While resolving toolchains for target @bazel_tools//tools/cpp:current_cc_toolchain: No matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type. Maybe --incompatible_use_cc_configure_from_rules_cc has been flipped and there is no default C++ toolchain added in the WORKSPACE file? See https://github.com/bazelbuild/bazel/issues/10134 for details and migration instructions.
ERROR: Analysis of target '//:android_dist' failed; build aborted: 
INFO: Elapsed time: 0.221s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 1336 targets configured)

comius · 2022-07-21T19:33:42Z

cc @katre Do you know anything on top of your head about this?

Building with: ``` USE_BAZEL_VERSION=last_green bazelisk build android_app --java_runtime_version=remotejdk_11 ``` reproduces bazelbuild/rules_java#64 Removing the `--java_runtime_version` argument does not reproduce it

keith · 2022-07-21T20:17:01Z

Here's a smaller repro case keith/bazel-rust-mobile-demo#3, passing --java_runtime_version=remotejdk_11 is what triggers things. Versus it defaulting to the local_jdk (maybe arm64 darwin specific?)

keith · 2022-07-21T20:24:11Z

Verified it repros on x86_64 macOS as well

jpsim · 2022-08-03T17:48:05Z

@comius @katre with the repro cases we've shared, do you have enough to go on to debug this? Envoy Mobile is temporarily unblocked by pointing to the branch from #65 but that's not something we want to keep long term.

jlaxson · 2023-03-15T00:42:07Z

I have a bunch of issues caused by the same problem, described at bazelbuild/bazel#17085. There's a reproducer there for several variants (wrong binaries used on exec platform, toolchain resolution failures that entirely break builds that use globs). @comius

RyanTheOptimist · 2023-05-11T16:52:21Z

This is still causing problem for the Envoy project and is blocking our move to bazel 6.2.0. Is there any more information we can provide to expedite this?

Work towards bazelbuild#17085 Work towards bazelbuild/rules_java#64 Split off from bazelbuild#18262

The `java_binary` Starlark implementation now only directly depends on a Java runtime toolchain when it is used as an executable rule. This work will eventually allow `java_binary` with `create_executable = False` to be compiled for platforms that do not provide a regular JDK, such as Android. Work towards bazelbuild#17085 Work towards bazelbuild/rules_java#64 Split off from bazelbuild#18262

This decouples the concepts of a Java runtime used to run on the target platform from the Java runtime that provides the bootstrap class path used during compilation. The former needs constraints on the target platform, whereas the latter should not have any constraints on the target platform to allow for cross-compilation to target platforms for which a JDK runtime is not available (e.g. Android). Work towards #17085 Work towards bazelbuild#64 Split off from #18262

The `java_binary` Starlark implementation now only directly depends on a Java runtime toolchain when it is used as an executable rule. This work will eventually allow `java_binary` with `create_executable = False` to be compiled for platforms that do not provide a regular JDK, such as Android. Work towards #17085 Work towards bazelbuild/rules_java#64 Split off from #18262 Closes #18620. PiperOrigin-RevId: 540884239 Change-Id: I3c06c02d1a9dc1be2775f409c393a386ac2260b7

Work towards #17085 Work towards bazelbuild/rules_java#64 Split off from #18262 Closes #18618. PiperOrigin-RevId: 541562617 Change-Id: I46a3057dcd5eec19cfd6f4ca6da2f76f0c3c2e3f

This decouples the concepts of a Java runtime used to run on the target platform from the Java runtime that provides the bootstrap class path used during compilation. The former needs constraints on the target platform, whereas the latter should not have any constraints on the target platform to allow for cross-compilation to target platforms for which a JDK runtime is not available (e.g. Android). Work towards #17085 Work towards bazelbuild#64 Split off from #18262

The `bootclasspath` rule in `rules_java` will soon use this toolchain type instead of the regular Java runtime toolchain type, which naturally carries a constraint on the target platform. Also adds more detailed explanations of the now three Java toolchain types. Work towards #17085 Work towards #18265 Work towards bazelbuild/rules_java#64 Split off from #18262 Closes #18841. PiperOrigin-RevId: 545756139 Change-Id: Ib9dd7a1c20c32315375722d6a023a89859daea9c

This decouples the concepts of a Java runtime used to run on the target platform from the Java runtime that provides the bootstrap class path used during compilation. The former needs constraints on the target platform, whereas the latter should not have any constraints on the target platform to allow for cross-compilation to target platforms for which a JDK runtime is not available (e.g. Android). Work towards #17085 Work towards bazelbuild#64 Split off from #18262

BEGIN_PUBLIC Copybara import of the project: -- cb82ccb by Fabian Meumertzheim <[email protected]>: Obtain bootstrap class path from dedicated runtime toolchain type This decouples the concepts of a Java runtime used to run on the target platform from the Java runtime that provides the bootstrap class path used during compilation. The former needs constraints on the target platform, whereas the latter should not have any constraints on the target platform to allow for cross-compilation to target platforms for which a JDK runtime is not available (e.g. Android). Work towards #17085 Work towards #64 Split off from #18262 -- ab3c792 by Fabian Meumertzheim <[email protected]>: Register toolchains in MODULE.bazel -- f2218a4 by Fabian Meumertzheim <[email protected]>: Update Bazel to 7.0.0-pre.20230710.5 -- 0294790 by Fabian Meumertzheim <[email protected]>: Add missing WORKSPACE `register_toolchains` calls END_PUBLIC COPYBARA_INTEGRATE_REVIEW=#114 from fmeum:17085-bootstrap-toolchain 0294790 PiperOrigin-RevId: 554446834 Change-Id: I961944be5bfe461528e94c85247f6c36236a561d

The `bootclasspath` rule in `rules_java` will soon use this toolchain type instead of the regular Java runtime toolchain type, which naturally carries a constraint on the target platform. Also adds more detailed explanations of the now three Java toolchain types. Work towards bazelbuild#17085 Work towards bazelbuild#18265 Work towards bazelbuild/rules_java#64 Split off from bazelbuild#18262 Closes bazelbuild#18841. PiperOrigin-RevId: 545756139 Change-Id: Ib9dd7a1c20c32315375722d6a023a89859daea9c

The `bootclasspath` rule in `rules_java` will soon use this toolchain type instead of the regular Java runtime toolchain type, which naturally carries a constraint on the target platform. Also adds more detailed explanations of the now three Java toolchain types. Work towards #17085 Work towards #18265 Work towards bazelbuild/rules_java#64 Split off from #18262 Closes #18841. PiperOrigin-RevId: 545756139 Change-Id: Ib9dd7a1c20c32315375722d6a023a89859daea9c Fixes #19201

The Java rules used toolchains of type `@bazel_tools//tools/jdk:runtime_toolchain_type` for two different purposes requiring different exec/target platform constraints, which led to issues when cross-compiling: The runtime added to the runfiles of an executable Java target has to have constraints on the target platform, whereas the runtime used to extract the bootclasspath should not have constraints on the target platform. As a result: 1. `@local_jdk` did not define any target constraints for its runtime, which allowed the runtime to provide the bootclasspath required by Android rules, but could also end up building `java_binary` targets that wouldn't run on the target (see bazelbuild#18265). 2. Remote JDKs defined a target constraint for the their runtimes, which prevented them from being added to the runfiles of targets they couldn't run on, but broke Android compilation due to the failure to resolve a runtime for bootclasspath extraction (see bazelbuild#17085). This is resolved by adding a third toolchain type, `bootstrap_runtime_toolchain_type`, that is only used by the `bootclasspath` rule (realizes bazelbuild#17085 (comment)). Detailed explanations of the different types and their intended constraints have been added to `@bazel_tools//tools/jdk:BUILD`. Addressing the cross-compilation errors then required the following changes: 1. Direct dependencies on the Java runtime have been removed from rules that do not need them (e.g. `android_library` and `java_binary` with `create_executable = False`). 3. Implicit dependencies on the Java runtime through the `bootclasspath` rule have been replaced with dependencies on the bootstrap runtime. 4. `{local,remote}_java_repository` now use the user-provided exec constraints of the Java toolchain as the target constraints of the associated runtime. 5. `@local_jdk` uses the auto-detected host constraints as exec constraints for the local Java toolchain. Fixes bazelbuild#17085 Fixes bazelbuild#18265 Fixes bazelbuild/rules_java#64 RELNOTES[INC]: Java runtime toolchains created via `local_java_repository` from `@bazel_tools//tools/jdk:local_java_repository.bzl`, which includes `local_jdk`, now have `target_compatible_with` set to the auto-detected host constraints. This can result in errors about toolchain resolution failures for `@bazel_tools//tools/jdk:runtime_toolchain_type`, especially when cross-compiling. These failures can be fixed in the following ways (listed in decreasing order of preference): * Replace `java_binary` targets that aren't meant to be run with `bazel run` or as tools during the build with `java_single_jar` (available in `@rules_java//java:java_single_jar.bzl`). Such targets do not require a Java runtime for the target configuration. * Set `--java_runtime_version=remotejdk_N` for some Java version `N` to let Bazel choose and download an appropriate remote JDK for the current target platform. This setting defaults to `local_jdk`, which means that Bazel can only use the local JDK, which isn't compatible with any other platform. * Manually define and register a `local_java_runtime` with no value set for `exec_compatible_with` (defaults to `[]`) and select it by setting `--java_runtime_version` to its `name`. This fully restores the previous behavior, but can result in incorrect results when cross-compiling (see bazelbuild#18265). Closes bazelbuild#18262. PiperOrigin-RevId: 574914446 Change-Id: I6cbfb7ffa2fbfd62e5f6fb49532b36be658dfa40

…enarios (#19900) The Java rules used toolchains of type `@bazel_tools//tools/jdk:runtime_toolchain_type` for two different purposes requiring different exec/target platform constraints, which led to issues when cross-compiling: The runtime added to the runfiles of an executable Java target has to have constraints on the target platform, whereas the runtime used to extract the bootclasspath should not have constraints on the target platform. As a result: 1. `@local_jdk` did not define any target constraints for its runtime, which allowed the runtime to provide the bootclasspath required by Android rules, but could also end up building `java_binary` targets that wouldn't run on the target (see #18265). 2. Remote JDKs defined a target constraint for the their runtimes, which prevented them from being added to the runfiles of targets they couldn't run on, but broke Android compilation due to the failure to resolve a runtime for bootclasspath extraction (see #17085). This is resolved by adding a third toolchain type, `bootstrap_runtime_toolchain_type`, that is only used by the `bootclasspath` rule (realizes #17085 (comment)). Detailed explanations of the different types and their intended constraints have been added to `@bazel_tools//tools/jdk:BUILD`. Addressing the cross-compilation errors then required the following changes: 1. Direct dependencies on the Java runtime have been removed from rules that do not need them (e.g. `android_library` and `java_binary` with `create_executable = False`). 3. Implicit dependencies on the Java runtime through the `bootclasspath` rule have been replaced with dependencies on the bootstrap runtime. 4. `{local,remote}_java_repository` now use the user-provided exec constraints of the Java toolchain as the target constraints of the associated runtime. 5. `@local_jdk` uses the auto-detected host constraints as exec constraints for the local Java toolchain. Fixes #17085 Fixes #18265 Fixes bazelbuild/rules_java#64 RELNOTES[INC]: Java runtime toolchains created via `local_java_repository` from `@bazel_tools//tools/jdk:local_java_repository.bzl`, which includes `local_jdk`, now have `target_compatible_with` set to the auto-detected host constraints. This can result in errors about toolchain resolution failures for `@bazel_tools//tools/jdk:runtime_toolchain_type`, especially when cross-compiling. These failures can be fixed in the following ways (listed in decreasing order of preference): * Replace `java_binary` targets that aren't meant to be run with `bazel run` or as tools during the build with `java_single_jar` (available in `@rules_java//java:java_single_jar.bzl`). Such targets do not require a Java runtime for the target configuration. * Set `--java_runtime_version=remotejdk_N` for some Java version `N` to let Bazel choose and download an appropriate remote JDK for the current target platform. This setting defaults to `local_jdk`, which means that Bazel can only use the local JDK, which isn't compatible with any other platform. * Manually define and register a `local_java_runtime` with no value set for `exec_compatible_with` (defaults to `[]`) and select it by setting `--java_runtime_version` to its `name`. This fully restores the previous behavior, but can result in incorrect results when cross-compiling (see #18265). Closes #18262. Commit f79ca02 PiperOrigin-RevId: 574914446 Change-Id: I6cbfb7ffa2fbfd62e5f6fb49532b36be658dfa40 Co-authored-by: Fabian Meumertzheim <[email protected]>

jpsim added a commit to jpsim/rules_java that referenced this issue Jul 21, 2022

Revert "Use target_compatible_with for Java runtimes"

4324398

This reverts commit 338c418. This is probably not the right fix, but it fixes bazelbuild#64 for me.

jpsim mentioned this issue Jul 21, 2022

Revert "Use target_compatible_with for Java runtimes" #65

Closed

comius self-assigned this Jul 21, 2022

keith mentioned this issue Jul 21, 2022

rules_java #64 repro case keith/bazel-rust-mobile-demo#3

Closed

This was referenced Jul 21, 2022

Update Envoy envoyproxy/envoy-mobile#2435

Merged

Android CI jobs are failing envoyproxy/envoy-mobile#2443

Closed

jpsim mentioned this issue Aug 18, 2022

Bump version to 5.1.0 #69

Merged

fmeum mentioned this issue May 2, 2023

Fix Java runtime toolchain resolution in cross-compilation scenarios bazelbuild/bazel#18262

Closed

phlax mentioned this issue May 11, 2023

Mobile android tests are permafailing since bazel 6.2 upgrade envoyproxy/envoy#27339

Closed

jpsim mentioned this issue May 11, 2023

[WIP/TESTING] mobile: Try updating rules_java envoyproxy/envoy#27342

Closed

fmeum added a commit to fmeum/bazel that referenced this issue Jun 8, 2023

Remove unnecessary Java runtime dependency of non-binary rules

4a02190

Work towards bazelbuild#17085 Work towards bazelbuild/rules_java#64 Split off from bazelbuild#18262

fmeum mentioned this issue Jun 8, 2023

Remove unnecessary Java runtime dependency of non-binary rules bazelbuild/bazel#18618

Closed

fmeum mentioned this issue Jun 9, 2023

Remove runtime dependency of non-executable java_binary bazelbuild/bazel#18620

Closed

fmeum mentioned this issue Jun 9, 2023

Obtain bootstrap class path from dedicated runtime toolchain type #114

Closed

fmeum mentioned this issue Jul 5, 2023

Add toolchain type for Java bootstrap runtime bazelbuild/bazel#18841

Closed

phlax mentioned this issue Aug 5, 2023

deps/mobile: Bump bazel -> 6.3.1, rules_java -> 6.4.0, io_bazel_rules_kotlin -> 1.8 envoyproxy/envoy#28854

Merged

iancha1992 mentioned this issue Aug 9, 2023

[Duplicate] Add toolchain type for Java bootstrap runtime bazelbuild/bazel#19216

Closed

fmeum mentioned this issue Aug 10, 2023

[6.4.0] Add toolchain type for Java bootstrap runtime bazelbuild/bazel#19220

Merged

bazel-io closed this as completed in bazel-io/bazel@f79ca02 Oct 19, 2023

iancha1992 mentioned this issue Oct 19, 2023

[7.0.0] Fix Java runtime toolchain resolution in cross-compilation scenarios bazelbuild/bazel#19900

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Question: How to find a JDK runtime with `master`? #64

Question: How to find a JDK runtime with `master`? #64

jpsim commented Jul 21, 2022

comius commented Jul 21, 2022

jpsim commented Jul 21, 2022 •

edited

Loading

comius commented Jul 21, 2022

keith commented Jul 21, 2022 •

edited

Loading

keith commented Jul 21, 2022

jpsim commented Aug 3, 2022

jlaxson commented Mar 15, 2023

RyanTheOptimist commented May 11, 2023

Question: How to find a JDK runtime with master? #64

Question: How to find a JDK runtime with master? #64

Comments

jpsim commented Jul 21, 2022

comius commented Jul 21, 2022

jpsim commented Jul 21, 2022 • edited Loading

comius commented Jul 21, 2022

keith commented Jul 21, 2022 • edited Loading

keith commented Jul 21, 2022

jpsim commented Aug 3, 2022

jlaxson commented Mar 15, 2023

RyanTheOptimist commented May 11, 2023

Question: How to find a JDK runtime with `master`? #64

Question: How to find a JDK runtime with `master`? #64

jpsim commented Jul 21, 2022 •

edited

Loading

keith commented Jul 21, 2022 •

edited

Loading