-
Notifications
You must be signed in to change notification settings - Fork 155
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make integration tests stricter #3183
Conversation
Does the PR have any schema changes?Does the PR have any schema changes?Looking good! No breaking changes found. Maintainer note: consult the runbook for dealing with any breaking changes. |
Looks like a fair number of tests does not refresh cleanly due to #2246 |
Another big issue affecting refresh tests is #3548 |
#3190 was concealed by lax testing options. |
https://github.com/pulumi/pulumi-aws/blob/master/examples/cloudwatchOidcManual/index.ts#L25 cycles with non-empty preview because the token changes between program invocations. Interesting question here on whether the token should persist into the statefile at all. |
395f4fe
to
756fed4
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Prior to this change, all example-based acceptance tests inherited a relaxed configuration.
This was masking issues with unexpected diffs during
refresh
and any possible regressions as there provider evolves. After the change, only the test that need it are configured with these relaxed options and the code is cross-correlated to root cause issues where appropriate.