Add API Keys #4804

upodroid · 2021-05-20T18:39:00Z

Fixes: hashicorp/terraform-provider-google#8959

If this PR is for Terraform, I acknowledge that I have:

Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

`google_apikeys_key`

modular-magician · 2021-05-20T18:40:23Z

Hello! I am a robot who works on Magic Modules PRs.

I have detected that you are a community contributor, so your PR will be assigned to someone with a commit-bit on this repo for initial review.

Thanks for your contribution! A human will be with you soon.

@melinath, please review this PR or find an appropriate assignee.

modular-magician · 2021-05-20T18:49:06Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 7 files changed, 822 insertions(+))
Terraform Beta: Diff ( 8 files changed, 824 insertions(+), 1 deletion(-))
TF Conversion: Diff ( 2 files changed, 15 insertions(+))

modular-magician · 2021-05-20T18:49:18Z

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=188484"

upodroid · 2021-05-20T19:04:54Z

There is a bug in the dclclient.

https://github.com/GoogleCloudPlatform/declarative-resource-client-library/blob/76bc5cc4eeee/services/google/apikeys/apikeys_utils.go#L54.

It doesn't check if the key exists before getting keyString.

==> Checking source code against gofmt...
==> Checking that code complies with gofmt requirements...
go generate  ./...
TF_ACC=1 TF_SCHEMA_PANIC_ON_ERROR=1 go test ./google -v -run=TestAccApikeysKey_basic -timeout 240m -ldflags="-X=github.com/hashicorp/terraform-provider-google/version.ProviderVersion=acc"
=== RUN   TestAccApikeysKey_basic
=== PAUSE TestAccApikeysKey_basic
=== CONT  TestAccApikeysKey_basic
2021/05/20 19:59:19 Beginning ApplyKey...
2021/05/20 19:59:19 [DEBUG][DCL INFO] User specified desired state: {
 Name: nil,
 Uid: nil,
 DisplayName: "key3ganyq94h2",
 KeyString: nil,
 CreateTime: nil,
 UpdateTime: nil,
 DeleteTime: nil,
 Restrictions: nil,
 Etag: nil,
 Project: "REDACTED",
}
2021/05/20 19:59:19 Fetching initial state...
2021/05/20 19:59:19 [DEBUG][DCL INFO] Google API Request: (id 5e1y5)
-----------[REQUEST]----------
GET /v2/projects/REDACTED/locations/global/keys/?alt=json HTTP/1.1
Host: apikeys.googleapis.com
User-Agent: Terraform/0.14.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google/acc DeclarativeClientLib/0.0.1
Content-Type: application/json
Accept-Encoding: gzip


-------[END REQUEST]--------
2021/05/20 19:59:19 [DEBUG] Retry Transport: starting RoundTrip retry loop
2021/05/20 19:59:19 [DEBUG] Retry Transport: request attempt 0
2021/05/20 19:59:19 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
GET /v2/projects/REDACTED/locations/global/keys/?alt=json HTTP/1.1
Host: apikeys.googleapis.com
User-Agent: Terraform/0.14.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google/acc DeclarativeClientLib/0.0.1
Content-Type: application/json
Accept-Encoding: gzip


-----------------------------------------------------
2021/05/20 19:59:21 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 200 OK
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Thu, 20 May 2021 18:59:22 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{}

-----------------------------------------------------
2021/05/20 19:59:21 [DEBUG] Retry Transport: Stopping retries, last request was successful
2021/05/20 19:59:21 [DEBUG] Retry Transport: Returning after 1 attempts
2021/05/20 19:59:21 [DEBUG][DCL INFO] Google API Response: (id 5e1y5) 
-----------[RESPONSE]----------
HTTP/2.0 200 OK
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Thu, 20 May 2021 18:59:22 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{}

-------[END RESPONSE]--------
2021/05/20 19:59:21 [DEBUG][DCL INFO] Google API Request: (id usnmt)
-----------[REQUEST]----------
GET /v2/projects/REDACTED/locations/global/keys//keyString?alt=json HTTP/1.1
Host: apikeys.googleapis.com
User-Agent: Terraform/0.14.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google/acc DeclarativeClientLib/0.0.1
Content-Type: application/json
Accept-Encoding: gzip


-------[END REQUEST]--------
2021/05/20 19:59:21 [DEBUG] Retry Transport: starting RoundTrip retry loop
2021/05/20 19:59:21 [DEBUG] Retry Transport: request attempt 0
2021/05/20 19:59:21 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
GET /v2/projects/REDACTED/locations/global/keys//keyString?alt=json HTTP/1.1
Host: apikeys.googleapis.com
User-Agent: Terraform/0.14.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google/acc DeclarativeClientLib/0.0.1
Content-Type: application/json
Accept-Encoding: gzip


-----------------------------------------------------
2021/05/20 19:59:21 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 400 Bad Request
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Thu, 20 May 2021 18:59:23 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "error": {
    "code": 400,
    "message": "\"projects/REDACTED/locations/global/keys/\" is not a valid resource name, it must be of the form \"projects/*/locations/global/keys/*\".",
    "status": "INVALID_ARGUMENT",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.PreconditionFailure",
        "violations": [
          {
            "type": "googleapis.com",
            "subject": "?error_code=900013&resource_name=projects/REDACTED/locations/global/keys/&resource_name_format=projects/*/locations/global/keys/*"
          }
        ]
      },
      {
        "@type": "type.googleapis.com/google.rpc.ErrorInfo",
        "reason": "API_SHARED_INVALID_RESOURCE_NAME",
        "domain": "apikeys.googleapis.com",
        "metadata": {
          "resource_name": "projects/REDACTED/locations/global/keys/",
          "resource_name_format": "projects/*/locations/global/keys/*"
        }
      }
    ]
  }
}

Also, the POST command can't take a name.

C02S62ZFFVH6:Git mahamed$ http POST https://apikeys.googleapis.com/v2/projects/REDACTED/locations/global/keys displayName=potato "Authorization: Bearer $(gcloud auth print-access-token)"
HTTP/1.1 200 OK
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Encoding: gzip
Content-Type: application/json; charset=UTF-8
Date: Thu, 20 May 2021 19:02:02 GMT
Server: ESF
Transfer-Encoding: chunked
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 0

{
    "name": "operations/akmf.p7-550924169191-1cc734ee-52a1-4be4-9d84-63cfd123c4c8"
}


C02S62ZFFVH6:Git mahamed$ http GET https://apikeys.googleapis.com/v2/projects/REDACTED/locations/global/keys "Authorization: Bearer $(gcloud auth print-access-token)"
HTTP/1.1 200 OK
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Encoding: gzip
Content-Type: application/json; charset=UTF-8
Date: Thu, 20 May 2021 19:03:43 GMT
Server: ESF
Transfer-Encoding: chunked
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 0

{
    "keys": [
        {
            "createTime": "2021-05-20T19:02:02.612654Z",
            "displayName": "potato",
            "etag": "DZ+rrFuuzTbvuFaUtCmE2g==",
            "name": "projects/550924169191/locations/global/keys/85cadc9a-ce18-40d0-ae10-7a1b3132a405",
            "uid": "85cadc9a-ce18-40d0-ae10-7a1b3132a405",
            "updateTime": "2021-05-20T19:02:02.717264Z"
        }
    ]
}

melinath · 2021-05-21T18:57:11Z

I don't have a lot of experience with the DCL yet so I'm reassigning review of this.

slevenick

Huh, yeah that's definitely wrong. Name seems to be expected but it's marked as readOnly: true.

I'll bring this up upstream

slevenick · 2021-05-21T22:41:31Z

tpgtools/api/apikeys/key.yaml

+          type: string
+          x-dcl-go-name: Name
+          x-kubernetes-immutable: true
+          readOnly: true


Did you copy this directly from the DCL source? It looks like this doesn't exist anymore: https://github.com/GoogleCloudPlatform/declarative-resource-client-library/blob/main/services/google/apikeys/key.yaml#L55

I wrote this by hand. I found that yaml file later while I was poking around the client internals. I tried that yaml file too and it returns the same error.

Ah, yeah those files should be what we use to generate the TF resource. Ideally we will link directly to those, but I think that has been harder than expected.

From what I can see it seems like the DCL resource expects name to be defined during creation, but I'm confused because it says output only

That is a bad assumption as some APIs won't have name field available until you do the 1st GET call after successful creation.

Some recent APIs I worked on:

https://cloud.google.com/storage-transfer/docs/reference/rest/v1/transferJobs/create (Perfect example as the name field is a random serverside generated value)

https://cloud.google.com/api-gateway/docs/reference/rest/v1beta/projects.locations.apis/create (Another example but the expected value that will be returned from the server is predictable)

Yeah, I'm familiar with the server generated name field. It looks like this API uses the keyId specified in the create request as the ending of the name field, so it should be possible to remote readOnly: true on name and specify it during a test.

The DCL normal flow for creating a resource is to check for existence before creating, which is failing because name is empty.

There is certainly something wrong with the OpenAPI spec in this case, as name should probably be marked required or renamed to keyId or something similar, and I'll get that figured out.

Cool, let me know when there is a fix available.

Should have a fix soon!

Any update on this?

upodroid · 2022-04-18T23:33:42Z

Fixed in #5637

add apikeys

2fefbaa

google-cla bot added the cla: yes label May 20, 2021

modular-magician requested a review from melinath May 20, 2021 18:40

melinath requested review from slevenick and removed request for melinath May 21, 2021 18:56

slevenick reviewed May 21, 2021

View reviewed changes

upodroid closed this Apr 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add API Keys #4804

Add API Keys #4804

upodroid commented May 20, 2021

modular-magician commented May 20, 2021

modular-magician commented May 20, 2021

modular-magician commented May 20, 2021

upodroid commented May 20, 2021

melinath commented May 21, 2021 •

edited

Loading

slevenick left a comment

slevenick May 21, 2021

upodroid May 21, 2021

slevenick May 21, 2021

upodroid May 21, 2021

upodroid May 21, 2021

slevenick May 21, 2021

upodroid May 21, 2021

upodroid Jun 21, 2021

slevenick Sep 1, 2021

upodroid Dec 1, 2021

upodroid commented Apr 18, 2022 •

edited

Loading

Add API Keys #4804

Add API Keys #4804

Conversation

upodroid commented May 20, 2021

modular-magician commented May 20, 2021

modular-magician commented May 20, 2021

Diff report:

modular-magician commented May 20, 2021

upodroid commented May 20, 2021

melinath commented May 21, 2021 • edited Loading

slevenick left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

upodroid commented Apr 18, 2022 • edited Loading

melinath commented May 21, 2021 •

edited

Loading

upodroid commented Apr 18, 2022 •

edited

Loading