CI check to check the results of CI tests

[CsatariGergely]

Is your feature request related to a problem? Please describe.

We would like to ensure that the quality of the open source component is guaranteed by the used CI tests.

Describe the solution you'd like

The CI check should check if actual tests are running in the CI and should check the results of the tests.

Describe alternatives you've considered

None

Additional context

There are two steps of this approach

1. Check if what is running is a test

For this we could check the name of workflows in a repo and identify if there is a workflow for testing (best idea to do this so far was to use a set of keywords). Maybe an alternative could be just to extend the keywords already checked.

2. Check the results of the tests

Verify that the execution of CheckRuns was successful for the previous N commits.

[CsatariGergely]

We are happy to create a pr if the idea works for the community.

[spencerschrock]

Can you describe what the short comings are of the current CI-Tests check are? Is your concern repos that use GitHub actions to label issues but don't have any tests?

check if actual tests are running in the CI and should check the results of the tests.

Step 1 is the big one here, I think Step 2 would fall into place easily after that.
Different repos structure and name their tests in all sorts of ways. With or without Makefiles (or similar), in workflows, in scripts, in re-usable workflows, etc.

[github-actions]

This issue has been marked stale because it has been open for 60 days with no activity.