Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

combine all access approval tests to run serially #4104

Conversation

megan07
Copy link
Contributor

@megan07 megan07 commented Oct 14, 2020

Fixes hashicorp/terraform-provider-google#7496

I think this should fix it, although I wasn't able to reproduce it locally. I think the problem is that if these tests are run in parallel, sometimes the organization is enrolled at the beginning of the project/folder tests (showing enrolled_ancestor) and then is no longer enrolled by the end of the test, so it's no longer returned. Running them serially should fix that.

If this PR is for Terraform, I acknowledge that I have:

  • Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
  • Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
  • Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
  • Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
  • Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)


@google-cla google-cla bot added the cla: yes label Oct 14, 2020
@modular-magician
Copy link
Collaborator

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 24 insertions(+), 3 deletions(-))
Terraform Beta: Diff ( 3 files changed, 24 insertions(+), 3 deletions(-))

@modular-magician
Copy link
Collaborator

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=152150"

@modular-magician
Copy link
Collaborator

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccProviderMeta_setModuleName|TestAccAccessApprovalSettings|TestAccComputeInstance_PTRRecord|TestAccComputeInstance_update|TestAccComputeInstance_secondaryAliasIpRange|TestAccComputeInstance_updateRunning_desiredStatusRunning_allowStoppingForUpdate|TestAccComputeInstance_updateRunning_desiredStatusNotSet_notAllowStoppingForUpdate|TestAccComputeInstance_updateRunning_desiredStatusRunning_notAllowStoppingForUpdate|TestAccComputeInstance_updateRunning_desiredStatusTerminated_allowStoppingForUpdate|TestAccComputeInstance_updateRunning_desiredStatusTerminated_notAllowStoppingForUpdate|TestAccComputeInstance_updateTerminated_desiredStatusNotSet_allowStoppingForUpdate|TestAccComputeInstance_updateTerminated_desiredStatusTerminated_allowStoppingForUpdate|TestAccComputeInstance_updateTerminated_desiredStatusNotSet_notAllowStoppingForUpdate|TestAccComputeInstance_updateTerminated_desiredStatusTerminated_notAllowStoppingForUpdate|TestAccComputeInstance_updateTerminated_desiredStatusRunning_allowStoppingForUpdate|TestAccComputeInstance_updateTerminated_desiredStatusRunning_notAllowStoppingForUpdate|TestAccComputeInstance_subnetworkUpdate You can view the result here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=152158"

@modular-magician
Copy link
Collaborator

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 23 insertions(+), 8 deletions(-))
Terraform Beta: Diff ( 3 files changed, 23 insertions(+), 8 deletions(-))

@modular-magician
Copy link
Collaborator

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=152166"

@modular-magician
Copy link
Collaborator

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccProviderMeta_setModuleName|TestAccAccessApprovalSettings You can view the result here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=152171"

@megan07 megan07 requested a review from c2thorn October 14, 2020 20:07
@@ -9,9 +9,7 @@ import (
"github.com/hashicorp/terraform-plugin-sdk/v2/terraform"
)

func TestAccAccessApprovalFolderSettings_update(t *testing.T) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a comment to these stating where to look for the test runner? I remember being confused for the AccessContextManagerServicePerimeter tests, but the comment really helped!

@modular-magician
Copy link
Collaborator

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 27 insertions(+), 8 deletions(-))
Terraform Beta: Diff ( 3 files changed, 27 insertions(+), 8 deletions(-))

@modular-magician
Copy link
Collaborator

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=152189"

@modular-magician
Copy link
Collaborator

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccProviderMeta_setModuleName You can view the result here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=152207"

@modular-magician
Copy link
Collaborator

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 29 insertions(+), 8 deletions(-))
Terraform Beta: Diff ( 3 files changed, 29 insertions(+), 8 deletions(-))

@modular-magician
Copy link
Collaborator

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=152311"

@modular-magician
Copy link
Collaborator

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccProviderMeta_setModuleName You can view the result here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=152317"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TestAccAccessApprovalProjectSettings_update is flaky
3 participants