Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

java_test "broken" with no guidance how to fix #1017

Closed
pmbethe09 opened this issue Mar 5, 2016 · 17 comments
Closed

java_test "broken" with no guidance how to fix #1017

pmbethe09 opened this issue Mar 5, 2016 · 17 comments

Comments

@pmbethe09
Copy link
Member

191d980

Apparently can use --legacy_bazel_java_test but there was no clear guidance how to fix rules of the form:
java_test(srcs = glob(["*Test.java"]), ...) with multiple Tests in one rule.

@damienmg
Copy link
Contributor

damienmg commented Mar 5, 2016

What's your test suite class? You add a main_class before didn't you, just change this main_class for a test_class should works.

@damienmg
Copy link
Contributor

damienmg commented Mar 6, 2016

Note to myself: fix the relnotes when doing the next bazel release

@damienmg
Copy link
Contributor

damienmg commented Mar 6, 2016

Also the link to the ml announcement: https://groups.google.com/forum/#!topic/bazel-discuss/r5Qbn1c7eg4

@damienmg damienmg self-assigned this Mar 6, 2016
@pmbethe09
Copy link
Member Author

I never used a TestSuite or main_class before. The previous behavior "just worked".

See attached mini example.
bazel test javatests/... --legacy_bazel_java_test
works just fine. But without the legacy flag:

"Class not found: [tests]"

Example.zip

@damienmg
Copy link
Contributor

damienmg commented Mar 7, 2016

The glob in the example resolve to only 1 file which is why the legacy behavior where finding the rule. The new behavior use the name of the rule to find the main class (compared to using the srcs if only one file is present like for java_binary). This is the behavior we use since the beginning at Google.

@pmbethe09
Copy link
Member Author

Add another class or 2 to blah and the result is still the same.
On Mar 7, 2016 10:37 AM, "Damien Martin-Guillerez" [email protected]
wrote:

The glob in the example resolve to only 1 file which is why the legacy
behavior where finding the rule. The new behavior use the name of the rule
to find the main class (compared to using the srcs if only one file is
present like for java_binary). This is the behavior we use since the
beginning at Google.


Reply to this email directly or view it on GitHub
#1017 (comment).

@damienmg
Copy link
Contributor

damienmg commented Mar 8, 2016

humm maybe we just take the first one, would be weird. We definitely don't create a test suite out of it.

@damienmg
Copy link
Contributor

damienmg commented Mar 8, 2016

(PS: the glob has to resolve to multiple files)

@pmbethe09
Copy link
Member Author

I don't know why it worked, but I can tell you that it did and picked up all of the tests in the glob.
See attached updated example -- run it then cat bazel-out/local_darwin-fastbuild/testlogs/javatests/tests/test.log
(or change target to linux)

I see
"
JUnit version 4.12
.Bar
.Foo

Time: 0.004

OK (2 tests)
"

It appears that org.junit.runner.JUnitCore was added as the main_class when none was specified, and that picked up all of the tests in the first jar 'javatests' jar file on the classpath.
This was easily be confirmed via experimentation/printlns

However, adding this class as the test_class and running without the '--legacy_bazel_java_test', gets me to a new failure "

  1. initializationError(org.junit.runner.JUnitCore)
    java.lang.Exception: No runnable methods
    "

@pmbethe09
Copy link
Member Author

and attachment
Example.zip

@damienmg
Copy link
Contributor

damienmg commented Mar 8, 2016

The test_class should be a test class. not the JUnitCore runner.

Oh yes the legacy behavior allow multiple test file in the srcs. Humm... not sure we should do anything there.

@pmbethe09
Copy link
Member Author

Easy enough to write a genrule to take my SRCS and generate
an org.junit.runners.Suite, but as I wrote, sad that this just worked
before, and now I have to add a layer of my own.

On Mon, Mar 7, 2016 at 10:19 PM, Damien Martin-Guillerez <
[email protected]> wrote:

The test_class should be a test class. not the JUnitCore runner.

Oh yes the legacy behavior allow multiple test file in the srcs. Humm...
not sure we should do anything there.


Reply to this email directly or view it on GitHub
#1017 (comment).

@pmbethe09
Copy link
Member Author

If anyone else runs into this -- I created a junit_tests skylark rule which assumes the srcs are all Junit tests, and creates a native.java_test using a generated test_class.
(attached)
junit.txt

damienmg pushed a commit that referenced this issue Mar 21, 2016
Baseline: 19b5675

Incompatible changes:

  - Skylark rules that are available from their own repository will
    now issue a warning when accessed through @bazel_tools.
  - Set --legacy_bazel_java_test to off by default. java_test will
    now have a slightly different behaviour, correctly emitting XML
    file but, as a downside, it needs correct declaration of the
    test suite (see #1017).
  - Labels in .bzl files in remote repositories will be resolved
    relative to their repository (instead of the repository the
    Skylark rule is used in).
  - Renamed proto_java_library to java_proto_library.  The former
    is now deprecated and will print out a warning when used.
  - android_sdk now compiles android_jack on the fly from
    android_jar, which means android_jar must be a jar and
    android_jack is now deprecated. The Jack tools (jack, jill,
    resource_extractor) must be specified.
  - Any project that depended on the objc_options rule will be
    broken. Can be fixed by adding attrs (infoplists,copts) directly
    to rules depending on the options.
  - .aidl files correctly require import statements for types
    defined in the same package and the same android_library.

New features:

  - Experimental Windows support is available.
  - Experimental support for writing remote repository rules in
    Skylark is available.
  - iOS ipa_post_processor attribute allows for user-defined IPA
    edits.
  - Adds a to_json method to Skylark structs, providing conversion to
    JSON format.
  - Native python rule can depend on skylark rule as long as skylark
    rule provides 'py' provider.
  - When using both --verbose_failures and --sandbox_debug, Bazel
    prints instructions how to spawn a debugging shell inside the
    sandbox.
  - add flag --sandbox_add_path, which takes a list of additional
    paths as argument and mount these paths to sandbox.

Important changes:

  - @androidsdk//:org_apache_http_legacy added for the legacy Apache
    classes for android sdk version 23 and above.
  - Genrules correctly work when used with bazel run.
  - When namespace-sandbox is run with the -D (debug) flag and
    inside a terminal, it spawns a shell inside the sandbox to aid in
    debugging when the sandboxed command fails.
  - Added --artifact to workspace generator for generating workspace
    and build file rules from artifact coodrinates.
  - Specifying --experimental_android_resource_shrinking on the
    command line will enable a resource shrinking pass on
    android_binary targets that already use Proguard.
  - J2ObjC updated to 1.0.1 release.
  - Added "root_symlinks" and "symlinks" parameters to Skylark
    runfiles() method.
  - You can no longer use objc_binary targets for the xctest_app
    attribute of an ios_test rule.
  - Enable overriding jsonnet binaries and stdlib for Jsonnet rules.
  - mount target of /etc/resolv.conf if it is a symlink.
  - Tests that failed to build because execution was halted no longer
    print their status.
  - Bazel warns if a cc rule's includes attribute contains up-level
    references that escape its package.
  - Add repository_ctx.download and repository_ctx.download_and_extract
    function.
bazel-io pushed a commit that referenced this issue Mar 31, 2016
Baseline: 19b5675

Incompatible changes:

  - Skylark rules that are available from their own repository will
    now issue a warning when accessed through @bazel_tools.
  - Set --legacy_bazel_java_test to off by default. java_test will
    now have a slightly different behaviour, correctly emitting XML
    file but, as a downside, it needs correct declaration of the
    test suite (see #1017).
  - Labels in .bzl files in remote repositories will be resolved
    relative to their repository (instead of the repository the
    Skylark rule is used in).
  - Renamed proto_java_library to java_proto_library.  The former
    is now deprecated and will print out a warning when used.
  - android_sdk now compiles android_jack on the fly from
    android_jar, which means android_jar must be a jar and
    android_jack is now deprecated. The Jack tools (jack, jill,
    resource_extractor) must be specified.
  - Any project that depended on the objc_options rule will be
    broken. Can be fixed by adding attrs (infoplists,copts) directly
    to rules depending on the options.
  - .aidl files correctly require import statements for types
    defined in the same package and the same android_library.

New features:

  - Experimental Windows support is available.
  - Experimental support for writing remote repository rules in
    Skylark is available.
  - iOS ipa_post_processor attribute allows for user-defined IPA
    edits.
  - Adds a to_json method to Skylark structs, providing conversion to
    JSON format.
  - Native python rule can depend on skylark rule as long as skylark
    rule provides 'py' provider.
  - When using both --verbose_failures and --sandbox_debug, Bazel
    prints instructions how to spawn a debugging shell inside the
    sandbox.
  - add flag --sandbox_add_path, which takes a list of additional
    paths as argument and mount these paths to sandbox.

Important changes:

  - @androidsdk//:org_apache_http_legacy added for the legacy Apache
    classes for android sdk version 23 and above.
  - Genrules correctly work when used with bazel run.
  - When namespace-sandbox is run with the -D (debug) flag and
    inside a terminal, it spawns a shell inside the sandbox to aid in
    debugging when the sandboxed command fails.
  - Added --artifact to workspace generator for generating workspace
    and build file rules from artifact coodrinates.
  - Specifying --experimental_android_resource_shrinking on the
    command line will enable a resource shrinking pass on
    android_binary targets that already use Proguard.
  - J2ObjC updated to 1.0.1 release.
  - Added "root_symlinks" and "symlinks" parameters to Skylark
    runfiles() method.
  - You can no longer use objc_binary targets for the xctest_app
    attribute of an ios_test rule.
  - Enable overriding jsonnet binaries and stdlib for Jsonnet rules.
  - mount target of /etc/resolv.conf if it is a symlink.
  - Tests that failed to build because execution was halted no longer
    print their status.
  - Bazel warns if a cc rule's includes attribute contains up-level
    references that escape its package.
  - Add repository_ctx.download and repository_ctx.download_and_extract
    function.
anupcshan pushed a commit to dropbox/bazel that referenced this issue Apr 5, 2016
Baseline: 19b5675

Incompatible changes:

  - Skylark rules that are available from their own repository will
    now issue a warning when accessed through @bazel_tools.
  - Set --legacy_bazel_java_test to off by default. java_test will
    now have a slightly different behaviour, correctly emitting XML
    file but, as a downside, it needs correct declaration of the
    test suite (see bazelbuild#1017).
  - Labels in .bzl files in remote repositories will be resolved
    relative to their repository (instead of the repository the
    Skylark rule is used in).
  - Renamed proto_java_library to java_proto_library.  The former
    is now deprecated and will print out a warning when used.
  - android_sdk now compiles android_jack on the fly from
    android_jar, which means android_jar must be a jar and
    android_jack is now deprecated. The Jack tools (jack, jill,
    resource_extractor) must be specified.
  - Any project that depended on the objc_options rule will be
    broken. Can be fixed by adding attrs (infoplists,copts) directly
    to rules depending on the options.
  - .aidl files correctly require import statements for types
    defined in the same package and the same android_library.

New features:

  - Experimental Windows support is available.
  - Experimental support for writing remote repository rules in
    Skylark is available.
  - iOS ipa_post_processor attribute allows for user-defined IPA
    edits.
  - Adds a to_json method to Skylark structs, providing conversion to
    JSON format.
  - Native python rule can depend on skylark rule as long as skylark
    rule provides 'py' provider.
  - When using both --verbose_failures and --sandbox_debug, Bazel
    prints instructions how to spawn a debugging shell inside the
    sandbox.
  - add flag --sandbox_add_path, which takes a list of additional
    paths as argument and mount these paths to sandbox.

Important changes:

  - @androidsdk//:org_apache_http_legacy added for the legacy Apache
    classes for android sdk version 23 and above.
  - Genrules correctly work when used with bazel run.
  - When namespace-sandbox is run with the -D (debug) flag and
    inside a terminal, it spawns a shell inside the sandbox to aid in
    debugging when the sandboxed command fails.
  - Added --artifact to workspace generator for generating workspace
    and build file rules from artifact coodrinates.
  - Specifying --experimental_android_resource_shrinking on the
    command line will enable a resource shrinking pass on
    android_binary targets that already use Proguard.
  - J2ObjC updated to 1.0.1 release.
  - Added "root_symlinks" and "symlinks" parameters to Skylark
    runfiles() method.
  - You can no longer use objc_binary targets for the xctest_app
    attribute of an ios_test rule.
  - Enable overriding jsonnet binaries and stdlib for Jsonnet rules.
  - mount target of /etc/resolv.conf if it is a symlink.
  - Tests that failed to build because execution was halted no longer
    print their status.
  - Bazel warns if a cc rule's includes attribute contains up-level
    references that escape its package.
  - Add repository_ctx.download and repository_ctx.download_and_extract
    function.
@ittaiz
Copy link
Member

ittaiz commented May 18, 2016

@damienmg does this mean you can't have a simple test target which runs all tests in the package?

@alltom
Copy link

alltom commented Aug 7, 2016

It seems like the documentation still needs to be updated. http://www.bazel.io/docs/be/java.html#java_test_examples has an example that just globs *.java, and from what I understand in this thread (and trying it on my own), that no longer works.

@darkcrawler01
Copy link

Please update the documentation

@wcurrie
Copy link

wcurrie commented Nov 18, 2018

Looking at how bazel uses java_test internally, the approach appears to be:

  1. define a junit Suite that finds tests by classpath scanning. eg AllTests
  2. build it using a java_library rule. eg test_runner
  3. include test_runner as a dep in individual java_test rules, listing the Suite from (1) as the test_class

java_test(
name = "cmdline_test",
size = "small",
srcs = glob(["cmdline/*.java"]),
tags = [
"foundations",
],
test_class = "com.google.devtools.build.lib.AllTests",
deps = [
":guava_junit_truth",
":test_runner",
":testutil",
"//src/main/java/com/google/devtools/build/lib/cmdline",
"//src/main/java/com/google/devtools/build/lib/vfs:pathfragment",
],
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants