Tests/Images missing in HTML report

[sergio-sastre]

If I run tests for android views
./gradlew :recyclerviewscreen:roborazzi:recordRoborazziDebug --tests '*Activity*'

The HTML report is empty

Although 2 of them are executed and I can see the screenshots in the corresponding folder

This does not happen in a module with Composables...
However, if I filter tests, something happens

1. run first tests for Activities only:
   ./gradlew :lazycolumnscreen:roborazzi:recordRoborazziDebug --tests '*Activity*'
   Everything is fine
2. run tests for Composables only:
   ./gradlew :lazycolumnscreen:roborazzi:recordRoborazziDebug --tests '*Composable*'

Then, the report lists also the Activity tests, which were filtered out in the last execution, but the images are missing:

I would not expect them to be in the list at all, since they were not executed in the last run

Also, some screenshots are not fully displayed in the preview of the report, althugh that is a minor issue

[takahirom]

Thanks for reporting this issue as always!
Roborazzi creates a report with each captureRoboImage() call and collects them at the end.
It's possible that reports from the previous test are still remaining and might have been collected. Do you think clearing them before executing the test would solve this issue?

[sergio-sastre]

Thanks for reporting this issue as always!
Roborazzi creates a report with each captureRoboImage() call and collects them at the end.
It's possible that reports from the previous test are still remaining and might have been collected. Do you think clearing them before executing the test would solve this issue?

If it’d work it’d be for the case where not executed tests are listed. However, I’m not sure. I was deleting the autogenerated folder with the reports and still getting missing images, what would theoretically have the same result as your suggestion 🤔

I suggest to create some examples in this repo and try to reproduce it by executing 2 different subsets of tests with filtering options and then see whether cleaning up programatically before the executions solves it, because what you mention makes sense to me

Regarding the empty report for Views, I will investigate a bit further… maybe there is sth I am missing, but I can ensure the captureRoboImage() methods are called there too 🧐. Would likely be a different bug too

[takahirom]

I've fixed the issue of keeping the previous test reports. Could you take a look at it?
https://github.com/takahirom/roborazzi/releases/tag/1.8.0-alpha-3

[sergio-sastre]

I've fixed the issue of keeping the previous test reports. Could you take a look at it?
https://github.com/takahirom/roborazzi/releases/tag/1.8.0-alpha-3

Let me try this evening. Thanks for your efforts as usual! ❤️

[sergio-sastre]

I've fixed the issue of keeping the previous test reports. Could you take a look at it?
https://github.com/takahirom/roborazzi/releases/tag/1.8.0-alpha-3

I can confirm both problems are gone (also the report without any screenshot) in 1.8.0-alpha-3. Great job!

However, not sure whether these are bugs or feature request:

• if the build fails because a screenshot differs, the report stays the same as before. I was expecting the diff image in the report, not only in the outputs
• if I add a new screenshot, the Summary remains the same, and I would expect to show "Added 1".

Both are also happening in 1.7.0-rc-1, so they are not introduced by these changes.
I can open a feature request/bug report for both, since they are unrelated to the original issue

[takahirom]

Thanks. This is definitely a bug. I didn't notice it. It looks like doLast isn't working when the task fails.
https://github.com/takahirom/roborazzi/blob/main/include-build/roborazzi-gradle-plugin/src/main/java/io/github/takahirom/roborazzi/RoborazziPlugin.kt#L217

[takahirom]

We might have to use Finalizer tasks. 👀
https://docs.gradle.org/current/userguide/more_about_tasks.html#sec:finalizer_tasks

[takahirom]

@sergio-sastre
Thanks for finding the bug.
I've released the fix. Could you check if it works?
https://github.com/takahirom/roborazzi/releases/tag/1.8.0-alpha-4

[sergio-sastre]

@sergio-sastre
Thanks for finding the bug.
I've released the fix. Could you check if it works?
https://github.com/takahirom/roborazzi/releases/tag/1.8.0-alpha-4

I’ll do my best to find some time to check it out this weekend. Thanks for the fast bug-fixing 🚀

[sergio-sastre]

@sergio-sastre
Thanks for finding the bug.
I've released the fix. Could you check if it works?
https://github.com/takahirom/roborazzi/releases/tag/1.8.0-alpha-4

Yeah, it works! thanks for the bugfix!

[takahirom]

Thanks!