Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run relevant VCR tests in GA #13170

Open
melinath opened this issue Dec 5, 2022 · 8 comments
Open

Run relevant VCR tests in GA #13170

melinath opened this issue Dec 5, 2022 · 8 comments
Labels
fixit-technical-debt magician Issues and features related to our bespoke CI size/m technical-debt
Milestone
Near-Term Goals

Comments

@melinath
Copy link
Collaborator

melinath commented Dec 5, 2022 •
edited
Loading

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment or link the pull request to this issue.

Currently it is possible for us to introduce crashing bugs like #13097 through innocuous-seeming changes like GoogleCloudPlatform/magic-modules#6824.

This caused GA-only test failures due to some missing changes, which were resolved in GoogleCloudPlatform/magic-modules#6860.

This kind of issue is not caught by our current tests, but it's very bad for our users and causes extra load on maintainers to fix. We should automate testing for it by running GA VCR tests on PRs for relevant tests.
@melinath melinath changed the title Automate testing of crashes on provider upgrade Run relevant VCR tests in GA Dec 5, 2022
@melinath melinath added this to the Near-Term Goals milestone Dec 6, 2022
@melinath melinath added the size/m label Dec 6, 2022
@melinath
Copy link
Collaborator Author

melinath commented Dec 6, 2022

may be blocked by project split

@rileykarson
Copy link
Collaborator

Related: #9148

@melinath melinath mentioned this issue Jan 2, 2024
Closed
@melinath
Copy link
Collaborator Author

melinath commented Jan 2, 2024

#16661 would have been caught by this.

@melinath
Copy link
Collaborator Author

#17388 would have been caught by this (12 100% test failures)

@melinath
Copy link
Collaborator Author

GoogleCloudPlatform/magic-modules#9965 fixed an issue that could have been caught by this - patch behavior was not properly updated during a promotion.

@rileykarson rileykarson added the magician Issues and features related to our bespoke CI label Mar 28, 2024
@melinath
Copy link
Collaborator Author

melinath commented May 10, 2024 •
edited
Loading

#18091 and #18092 would have been caught by this. (7 100% failures)

@melinath
Copy link
Collaborator Author

#18239 would have been caught by this.

@melinath
Copy link
Collaborator Author

melinath commented Aug 9, 2024 •
edited
Loading

The following would have been caught by this:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
fixit-technical-debt magician Issues and features related to our bespoke CI size/m technical-debt
Projects
None yet
Milestone
Near-Term Goals
Development

No branches or pull requests
2 participants
@melinath @rileykarson