Testing JUnit parser for GH Actions #16966
Conversation
|
@chia7712 here is the junit parser PR. It parses all the JUnit reports and gathers the failures and skipped tests. These are printed into a Markdown table that becomes part of the job summary. https://github.com/apache/kafka/actions/runs/10514796991?pr=16966#summary-29133975884 Screenshot: The script also exits with 1 if any failures are found, which gives us the failing status check in the PR. |
.github/workflows/pr.yml
Outdated
| name: junit-reports-${{ matrix.java }} | ||
| path: | | ||
| **/test-results/**/*.xml | ||
| **/build/reports/tests/test/index.html |
There was a problem hiding this comment.
If we archive only the index.html, developers can't trace the "error stack" from the index.html as the linked class page is nonexistent. Maybe we should cache whole folder of /build/reports/tests/test/ instead of /test-results/**/*.xml?
.github/scripts/junit.py
Outdated
| logger.debug(f"Found skipped test: {skipped_test}") | ||
| table.append(("⚠️", simple_class_name, skipped_test.test_name, "Skipped", "")) | ||
| duration = pretty_time_duration(total_time) | ||
| print(f"{total_tests} tests run in {duration}, {total_failures} failed ❌, {total_skipped} skipped ⚠️, {total_errors} errors.") |
There was a problem hiding this comment.
Could we add the artifact url link? contributors can download the total reports when they check this great summary? the link (actions/upload-artifact#50 (comment)) shows how to get the artifact url and we can pass it through env. WDYT?
There was a problem hiding this comment.
Yea, that seems useful. Especially since we run two tests suites, giving a link to the correct archive is nice.
A few improvements for JUnit in the Actions workflow: * Generate a human readable job summary of the tests * Fail the workflow if JUnit tests fail * Archive the HTML JUnit reports Reviewers: Chia-Ping Tsai <[email protected]>
A few improvements for JUnit in the Actions workflow: * Generate a human readable job summary of the tests * Fail the workflow if JUnit tests fail * Archive the HTML JUnit reports Reviewers: Chia-Ping Tsai <[email protected]>
This patch adds a script to parse the JUnit XML files and produce a job summary. The JUnit test reports are also archived by this patch.