Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JUnit TestTemplate tests are not re-run on code change in dev mode #40770

Open
holly-cummins opened this issue May 22, 2024 · 0 comments
Open
Labels
area/devmode kind/bug Something isn't working

Comments

@holly-cummins
Copy link
Contributor

holly-cummins commented May 22, 2024

Describe the bug

I spotted two issues with @TestTemplate and dev mode, while adding tests to improve our coverage in this area.

Problem 1: Changing a file covered by a templated test does not trigger a re-run of the templated test.

Problem 2: Failures in templated tests are under-counted; in my test, when things pass, there are two passes, but when things fail, there's only one failure. (In normal mode, the expected two failures are reported, so this is a dev and test-mode thing.)

How to Reproduce?

The disabled testThatChangesTriggerRerunsOfJUnitTestTemplates test in DevMojoIT reproduces the problem. (PR.)

It's also possible to reproduce manually (once that PR merges).

  1. Run the maven tests (this just does the substitutions in the test projects): mvn -f integration-tests/maven -Dtest=DevMojoIT clean verify
  2. In one terminal, run the test application cd integration-tests/maven/target/test-classes/projects/test-template-processed && quarkus dev. It should report two tests passing.
  3. In a separate window, edit the java file for the generated test project: vi integration-tests/maven/target/test-classes/projects/test-template-processed/src/main/java/org/acme/HelloResource.java. Change "hello" to something else.
  4. Notice that the tests do not re-run. This is bad.
  5. If you quit quarkus and then re-run quarkus dev, you'll get failure (which is good), but only one (which is unexpected).

When reproducing, these tests can sometimes get into an unclean state where they fail with Wrong Name linker/compiler errors, and in those cases rm -rf integration-tests/maven/target/test-classes/projects/*/target can fix it (a maven clean doesn't clean those out).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/devmode kind/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant