[improve][CI] Replace test reporting with less verbose solution

[lhotari]

Fixes #16736

Motivation

See #16736

This PR reduces noise in test reporting and removes the misleading check runs from PR checks.

Modifications

• remove separate workflow that added GitHub Actions check runs
• replaces test reporting with https://github.com/mikepenz/action-junit-report which has been
  copied to https://github.com/apache/pulsar-test-infra/tree/master/action-junit-report
• doc-not-needed

[lhotari]

@tisonkun @maxsxu please join the review too

[tisonkun]

Generally looks good. Thanks!

Two comments:

1. ./.github/actions/merge-test-reports seems no more references, we may remove it.
2. I don't find a report on the new progress in the demo [improve][CI] Replace test reporting with less verbose solution lhotari/pulsar#78, can you show me a direct link for the test report page? (IIUC it should be something like this one: https://github.com/apache/pulsar/pull/17062/checks?check_run_id=8027361638)

[lhotari]

Generally looks good. Thanks!

Two comments:

1. ./.github/actions/merge-test-reports seems no more references, we may remove it.

Yes makes sense.

2. I don't find a report on the new progress in the demo [improve][CI] Replace test reporting with less verbose solution lhotari/pulsar#78, can you show me a direct link for the test report page? (IIUC it should be something like this one: https://github.com/apache/pulsar/pull/17062/checks?check_run_id=8027361638)

@tisonkun
The goal of the PR is to not add the report as a check run. The report is displayed on the workflow run summary page: https://github.com/lhotari/pulsar/actions/runs/2943996610
Besides the markdown report, annotations are added with the relevant code location and stack trace.

[tisonkun]

Thanks for your explanation! I get the point. +1 to merge,

[tisonkun]

One thing to remind here is that we may want to cherry-pick this change to previous release branch. I suppose we can possibly remove the test-reporter action from pulsar-test-infra.

Or we can tag pulsar-test-infra and use a stable version :)

[lhotari]

One thing to remind here is that we may want to cherry-pick this change to previous release branch. I suppose we can possibly remove the test-reporter action from pulsar-test-infra.

Or we can tag pulsar-test-infra and use a stable version :)

The forks of apache/pulsar still depend on test-reporter action and it might be a good idea to keep it for some time, also after our maintenance branches no more use it.

btw. There would be a way in GitHub Actions to make maintenance branches use workflows defined in the master branch. This is possible with the new reuseable workflows, https://github.blog/changelog/2022-08-22-github-actions-improvements-to-reusable-workflows-2/ . There's a tradeoff in doing this, but that could make things easier if we have a single pipeline to maintain.

[maxsxu]

Thanks for this clean PR!

Just wondering how we added the action-junit-report against the pulsar-test-infra repo?

[tisonkun]

@maxsxu I think @lhotari pushed a commit apache/pulsar-test-infra@eeb9107 directly.

[lhotari]

@maxsxu I think @lhotari pushed a commit apache/pulsar-test-infra@eeb9107 directly.

yes

[maxsxu]

LGTM