Access Context Manager doesn't work with user ADC because terraform doesn't set the X-Goog-User-Project Header

[upodroid]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

root@DESKTOP-7JUP8RO:~$ terraform version
Terraform v0.12.24
+ provider.google v3.28.0

Affected Resource(s)

• google_access_context_manager_*

Terraform Configuration Files

provider "google" {
  region       = "europe-west6"
  version      = "~> 3.27"
  user_project_override = true
}

resource "google_access_context_manager_access_policy" "base" {
  parent = data.google_organization.org.name
  title  = "test"
}

data "google_organization" "org" {
  domain = "dummy.borg.dev"
}

Debug Output

a2020-07-05T11:43:16.759+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: ---[ REQUEST ]---------------------------------------
t2020-07-05T11:43:16.759+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: POST /v1/accessPolicies?alt=json HTTP/1.1
2020-07-05T11:43:16.759+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: Host: accesscontextmanager.googleapis.com
i2020-07-05T11:43:16.759+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: User-Agent: HashiCorp Terraform/0.12.24 (+https://www.terraform.io) Terraform Plugin SDK/1.11.0 terraform-provider-google/3.28.0
n2020-07-05T11:43:16.759+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: Content-Length: 55
2020-07-05T11:43:16.759+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: Content-Type: application/json
g...2020-07-05T11:43:16.759+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: Accept-Encoding: gzip
2020-07-05T11:43:16.759+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe:
2020-07-05T11:43:16.759+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: {
2020-07-05T11:43:16.759+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe:  "parent": "organizations/684510004702",
2020-07-05T11:43:16.759+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe:  "title": "test"

2020-07-05T11:43:16.759+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: }
2020-07-05T11:43:16.759+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe:
2020-07-05T11:43:16.759+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: -----------------------------------------------------
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: 2020/07/05 11:43:17 [DEBUG] Google API Response Details:
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: ---[ RESPONSE ]--------------------------------------
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: HTTP/1.1 403 Forbidden
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: Connection: close
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: Transfer-Encoding: chunked
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: Alt-Svc: h3-29=":443"; ma=2592000,h3-27=":443"; ma=2592000,h3-25=":443"; ma=2592000,h3-T050=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: Cache-Control: private
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: Content-Type: application/json; charset=UTF-8
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: Date: Sun, 05 Jul 2020 10:43:16 GMT
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: Server: ESF
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: Vary: Origin
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: Vary: X-Origin
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: Vary: Referer
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: X-Content-Type-Options: nosniff
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: X-Frame-Options: SAMEORIGIN
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: X-Xss-Protection: 0
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe:
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: 22a
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: {
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe:   "error": {
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe:     "code": 403,
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe:     "message": "Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the accesscontextmanager.googleapis.com. We recommend configuring the billing/quota_project setting in gcloud or using a service account through the auth/impersonate_service_account setting. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/.",
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe:     "status": "PERMISSION_DENIED"
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe:   }
2020-07-05T11:43:17.080+0100 [DEBUG] plugin.terraform-provider-google_v3.28.0_x5.exe: }

Expected Behavior

It should have worked if the ADC has specified a quota project. https://cloud.google.com/sdk/gcloud/reference/auth/application-default/set-quota-project

Actual Behavior

403 error because terraform doesn't set the X-Goog-User-Project header.

Steps to Reproduce

Generate an ADC and set the quota project. Then try to apply the terraform config above.

1. terraform apply

root@DESKTOP-7JUP8RO:~$ terraform apply --auto-approve
data.google_organization.org: Refreshing state...
google_access_context_manager_access_policy.base: Creating...

Error: Error creating AccessPolicy: googleapi: Error 403: Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the accesscontextmanager.googleapis.com. We recommend configuring the billing/quota_project setting in gcloud or using a service account through the auth/impersonate_service_account setting. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/.

  on main.tf line 1, in resource "google_access_context_manager_access_policy" "base":
   1: resource "google_access_context_manager_access_policy" "base" {

Important Factoids

Access Context Manager API works with ADCs as shown below. We should be able to set a quota project for organization resources that need it.

 ⨯ REDACTED@DESKTOP-7JUP8RO  ~   curl.exe -H "X-Goog-User-Project: REDACTED" -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" "https://accesscontextmanager.googleapis.com/v1/accessPolicies?parent=organizations/684510004702"
C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin\..\lib\third_party\google\auth\_default.py:69: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that most server applications use service accounts instead. If your application continues to use end user credentials from Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error. For more information about service accounts, see https://cloud.google.com/docs/authentication/
  warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)
{
  "accessPolicies": [
    {
      "name": "accessPolicies/893819325231",
      "parent": "organizations/684510004702",
      "title": "RestrictAPIs",
      "etag": "fa476d11acb0023b"
    }
  ]
}

https://qiita.com/nii_yan/items/3bcd2940e15486b4c6e2

The ADC has a quota project set as per the docs.

 ⨯ REDACTED@DESKTOP-7JUP8RO  ~  cat C:/Users/REDACTED/AppData/Roaming/gcloud/application_default_credentials.json
{
  "client_id": "764086051850-6qr4p6gpi6hn506pt8ejuq83di341hur.apps.googleusercontent.com",
  "client_secret": "REDACTED",
  "quota_project_id": "REDACTED",
  "refresh_token": "REDACTED",
  "type": "authorized_user"
}

[edwardmedia]

@upodroid to apply google_access_context_manager_access_policy, the account needs to have org level permission. Can you double checked if this is the reason? Also I am curious why you need to set a quote project here? Can you provision the resource by other tools such as gcloud with the quote project?

[upodroid]

The account executing the commands has the org level permissions. Issues #6401 and #3620 are similar to mine but now we know that accesscontextmanager API allows ADCs to work if the quota project is defined so this needs to be implemented in the terraform provider.

In the meanwhile, i'm using service account impersonation to carry on with my work.

[edwardmedia]

@upodroid I am glad you are not blocked and closing this issue then. If you want this feature to be implemented, would you please file an enhancement using below link and specify the reason why you think you need it? We could prioritize it into the backlog. Thank you
https://github.com/terraform-providers/terraform-provider-google/issues/new?template=enhancement.md

[upodroid]

I'll open an enhancement but just to clarify this is a bug that needs a fix. The API works with ADCs as I showed with curl above so terraform needs to implement the missing header.

 REDACTED@DESKTOP-7JUP8RO  ~\..\..\terraform-google-forseti   enable-firewall-logging ≣ ~15 -0 !  curl.exe  -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" "https://accesscontextmanager.googleapis.com/v1/accessPolicies?parent=organizations/684510004702"
C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin\..\lib\third_party\google\auth\_default.py:69: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that most server applications use service accounts instead. If your application continues to use end user credentials from Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error. For more information about service accounts, see https://cloud.google.com/docs/authentication/
  warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)
{
  "error": {
    "code": 403,
    "message": "Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the accesscontextmanager.googleapis.com. We recommend configuring the billing/quota_project setting in gcloud or using a service account through the 
auth/impersonate_service_account setting. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/.",
    "status": "PERMISSION_DENIED"
  }
}
 REDACTED@DESKTOP-7JUP8RO  ~\..\..\terraform-google-forseti   enable-firewall-logging ≣ ~1 -0 !  curl.exe -H "X-Goog-User-Project: REDACTED" -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" "https://accesscontextmanager.googleapis.com/v1/accessPolicies?parent=organizations/684510004702"
C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin\..\lib\third_party\google\auth\_default.py:69: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that most server applications use service accounts instead. If your application continues to use end user credentials from Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error. For more information about service accounts, see https://cloud.google.com/docs/authentication/
  warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)
{
  "accessPolicies": [
    {
      "name": "accessPolicies/893819325231",
      "parent": "organizations/684510004702",
      "title": "RestrictAPIs",
      "etag": "d1135c2e91171423"
    }
  ]
}

Thanks

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!