You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I spotted two issues with @TestTemplate and dev mode, while adding tests to improve our coverage in this area.
Problem 1: Changing a file covered by a templated test does not trigger a re-run of the templated test.
Problem 2: Failures in templated tests are under-counted; in my test, when things pass, there are two passes, but when things fail, there's only one failure. (In normal mode, the expected two failures are reported, so this is a dev and test-mode thing.)
It's also possible to reproduce manually (once that PR merges).
Run the maven tests (this just does the substitutions in the test projects): mvn -f integration-tests/maven -Dtest=DevMojoIT clean verify
In one terminal, run the test application cd integration-tests/maven/target/test-classes/projects/test-template-processed && quarkus dev. It should report two tests passing.
In a separate window, edit the java file for the generated test project: vi integration-tests/maven/target/test-classes/projects/test-template-processed/src/main/java/org/acme/HelloResource.java. Change "hello" to something else.
Notice that the tests do not re-run. This is bad.
If you quit quarkus and then re-run quarkus dev, you'll get failure (which is good), but only one (which is unexpected).
When reproducing, these tests can sometimes get into an unclean state where they fail with Wrong Name linker/compiler errors, and in those cases rm -rf integration-tests/maven/target/test-classes/projects/*/target can fix it (a maven clean doesn't clean those out).
The text was updated successfully, but these errors were encountered:
Describe the bug
I spotted two issues with
@TestTemplate
and dev mode, while adding tests to improve our coverage in this area.Problem 1: Changing a file covered by a templated test does not trigger a re-run of the templated test.
Problem 2: Failures in templated tests are under-counted; in my test, when things pass, there are two passes, but when things fail, there's only one failure. (In normal mode, the expected two failures are reported, so this is a dev and test-mode thing.)
How to Reproduce?
The disabled
testThatChangesTriggerRerunsOfJUnitTestTemplates
test inDevMojoIT
reproduces the problem. (PR.)It's also possible to reproduce manually (once that PR merges).
mvn -f integration-tests/maven -Dtest=DevMojoIT clean verify
cd integration-tests/maven/target/test-classes/projects/test-template-processed && quarkus dev
. It should report two tests passing.vi integration-tests/maven/target/test-classes/projects/test-template-processed/src/main/java/org/acme/HelloResource.java
. Change "hello" to something else.quarkus dev
, you'll get failure (which is good), but only one (which is unexpected).When reproducing, these tests can sometimes get into an unclean state where they fail with
Wrong Name
linker/compiler errors, and in those casesrm -rf integration-tests/maven/target/test-classes/projects/*/target
can fix it (a maven clean doesn't clean those out).The text was updated successfully, but these errors were encountered: