Unable to retrieve P4SA: [[email protected]] from GAIA. Could be GAIA propagation delay or request from deleted apps.

[mattelacchiato]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v0.14.5
+ provider registry.terraform.io/hashicorp/archive v2.0.0
+ provider registry.terraform.io/hashicorp/external v2.0.0
+ provider registry.terraform.io/hashicorp/google v3.54.0
+ provider registry.terraform.io/hashicorp/null v3.0.0
+ provider registry.terraform.io/hashicorp/random v2.3.1

Affected Resource(s)

• google_app_engine_standard_app_version

Terraform Configuration Files

Please exchange XXX with your values.

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 3.54"
    }
    random = {
      source  = "hashicorp/random"
      version = "~> 2.2"
    }
  }
  required_version = ">= 0.13"
}

provider "google" {
  region = "europe-west3"
}

resource "random_id" "main" {
  byte_length = 1
  prefix      = "mb-test-"
}

resource "google_project" "main" {
  name            = random_id.main.hex
  project_id      = random_id.main.hex
  billing_account = "XXX"
  org_id          = "XXX"
}

resource "google_project_service" "main-service" {
  for_each = toset([
    "appengine.googleapis.com",
    "iap.googleapis.com",
    "logging.googleapis.com",
  ])

  service = each.key

  project            = google_project.main.project_id
  disable_on_destroy = false
}

resource "google_iap_brand" "default" {
  support_email     = "[email protected]"
  application_title = "XXX Application"
  project           = google_project.main.project_id
  depends_on        = [google_project_service.main-service]
}

resource "google_iap_client" "default" {
  display_name = "Default Client"
  brand        = google_iap_brand.default.name
}


resource "google_app_engine_application" "gra" {
  project     = google_project.main.project_id
  location_id = "europe-west3"

  iap {
    enabled              = true
    oauth2_client_id     = google_iap_client.default.client_id
    oauth2_client_secret = google_iap_client.default.secret
  }
  # service appengine.googleapis.com needs to be enabled
  depends_on = [google_project_service.main-service]
}


resource "google_app_engine_standard_app_version" "gra_current" {
  project    = google_project.main.project_id
  version_id = "v1"
  service    = "default"
  runtime    = "python37"


  deployment {
    zip {
      source_url = "https://storage.googleapis.com/tf-delme-43243214231412/master.zip"
    }
  }

  # Cannot be empty in contrast to documentation, therefore set to a sane default taken from google documentation.
  # This needs to be in sync with the entrypoint definition in the app.yaml in the app itself.
  entrypoint {
    shell = "gunicorn -b :$PORT main:app"
  }

  # automatic scaling is the only option to explicity set AppEngine to not scale to zero (min_instances)
  automatic_scaling {
    min_idle_instances  = 1
    max_idle_instances  = 1
    min_pending_latency = "1s"
    max_pending_latency = "5s"
    standard_scheduler_settings {
      target_cpu_utilization        = 0.75
      target_throughput_utilization = 0.75
      min_instances                 = 0
      max_instances                 = 3
    }
  }

  # default service and last version can't be destroyed, will be deleted/suspended together with the project.
  noop_on_destroy = true

  # fails without the application being active
  depends_on = [google_app_engine_application.gra]
}

Debug Output

Please find the TRACE log here:
https://gist.github.com/mattelacchiato/969d5149c52d1d552696606329dec7dd

Expected Behavior

The standard app version should be created. If internal systems at google need more time for creation, terraform google provider should retry.

Actual Behavior

terraform apply fails:

Unable to retrieve P4SA: [[email protected]] from GAIA. Could be GAIA propagation delay or request from deleted apps.

Steps to Reproduce

1. terraform apply

Important Factoids

It may need some retries. On my machine, it took 2-3 retries (terraform apply - terraform destroy), until error was reproduced.

References

• This developer has the same problem: https://github.com/felipecoe/gcp#common-gcp-fails

[rileykarson]

We run into this most nights in our testing environment (TestAccAppEngineFlexibleAppVersion_update most recently, but multiple app engine tests appear to be affected)

We can probably add a retry rule for when this occurs.

[norbjd]

We faced the same issue when trying to upgrade a Cloud Composer environment (error during webserver deployment, which is deployed in App engine).

[melinath]

It looks like this is still happening on master. I'll need to investigate a bit more. https://ci-oss.hashicorp.engineering/buildConfiguration/GoogleCloud_ProviderGoogleCloudGoogleProject/170671

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!