Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scorecard: use TestList for output, exit 1 on test failure #3427

Merged
merged 2 commits into from
Jul 15, 2020

Conversation

joelanford
Copy link
Member

Description of the change:

  1. Instead of aggregating all test results from ALL scorecard tests under a single v1alpha3.Test status, use a new v1alpha3.TestList to output a list of v1alpha3.Test objects, one for each test defined in the scorecard config file.
  2. Exit with status 1 when any test fails.

Motivation for the change:

  1. The test results should not be disassociated from the test spec (image/entrypoint) that provided the result.
  2. To match existing documentation

Checklist

If the pull request includes user-facing changes, extra documentation is required:

@jmccormick2001
Copy link
Contributor

seems to work for the basic/olm case, the kuttl image needs the updates for that to work again, also the custom test image would need the same updates.

@joelanford
Copy link
Member Author

Just tested kuttl, it also works, which is expected because this PR does not change anything about the pod log interface and result gathering.

@joelanford joelanford merged commit 4d8960c into operator-framework:master Jul 15, 2020
@joelanford joelanford deleted the scorecard-output-api branch July 15, 2020 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants