Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

google_cloud_run_v2_service and google_cloud_run_v2_job do not support Direct VPC egress with a VPC network #15568

Comments

@le0pard
Copy link

le0pard commented Aug 18, 2023

Description

Screenshot 2023-08-19 at 01 45 23

Cloud Run v2 have in preview "Direct VPC egress", which based on this comparison https://cloud.google.com/run/docs/configuring/vpc-connect-comparison much better than "Serverless VPC Access connectors"

Problem, that "google_cloud_run_v2_service" in "vpc_access" support only "connector" for "Serverless VPC Access connectors" and "egress" settings. Even if I am try to skip "connector" and only have "egress", I will get error:

│ Error: Error updating Service "projects/tests/locations/us-central1/services/test-service": googleapi: Error 400: Violation in UpdateServiceRequest.service.template.vpc_access.connector: one and only one of connector and network_interfaces must be set.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.BadRequest",
│     "fieldViolations": [
│       {
│         "description": "one and only one of connector and network_interfaces must be set.",
│         "field": "Violation in UpdateServiceRequest.service.template.vpc_access.connector"
│       }
│     ]
│   }
│ ]
│
│   with google_cloud_run_v2_service.test_service,
│   on cloud_run.tf line 251, in resource "google_cloud_run_v2_service" "test_service":
│  251: resource "google_cloud_run_v2_service" "test_service" {

Same issue have "google_cloud_run_v2_job" resource.

More info about "Direct VPC egress": https://cloud.google.com/run/docs/configuring/vpc-direct-vpc

New or Affected Resource(s)

  • google_cloud_run_v2_service
  • google_cloud_run_v2_job

Potential Terraform Configuration

I cannot find API call attributes, but I see this payload part, which json payload send GCP UI

// ...
"vpcSettings": {
  "vpcAccessEgress": "private-ranges-only",
  "directVpc": {
    "vpcNetworkInterfaces": [
      {
        "network": "test-vpc",
        "subnetwork": "test-vpc-subnet",
        "tags": [
          "cloud-run"
        ]
      }
    ]
  }
}

References

b/298050505

@github-actions github-actions bot added forward/review In review; remove label to forward service/run labels Aug 18, 2023
@ScottSuarez ScottSuarez removed the forward/review In review; remove label to forward label Aug 25, 2023
@ScottSuarez ScottSuarez changed the title Resource google_cloud_run_v2_service and google_cloud_run_v2_job are not support Direct VPC egress with a VPC network google_cloud_run_v2_service and google_cloud_run_v2_job do not support Direct VPC egress with a VPC network Aug 25, 2023
@melinath melinath added this to the Goals milestone Aug 28, 2023
@mrak-
Copy link

mrak- commented Sep 7, 2023

Please tell me what is known about this problem? How can it be solved, maybe there is an example?

@le0pard
Copy link
Author

le0pard commented Sep 7, 2023

Hello, right now I solve this by YML file:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  ...
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/minScale: '1'
        autoscaling.knative.dev/maxScale: '4'
        run.googleapis.com/network-interfaces: '[{"network":"private-network","subnetwork":"subnet","tags":["net"]}]'
        run.googleapis.com/vpc-access-egress: private-ranges-only
        run.googleapis.com/cpu-throttling: 'true'
        run.googleapis.com/startup-cpu-boost: 'true'
        run.googleapis.com/sessionAffinity: 'false'
        run.googleapis.com/execution-environment: gen2
    spec:
      containerConcurrency: 20
      timeoutSeconds: '60'
      ...

Major parts here is "run.googleapis.com/network-interfaces" and "run.googleapis.com/vpc-access-egress"

and apply this changes by gcloud run services replace service.yml. Also this option available in UI (as I show in screenshot). But not options for terraform @mrak-

@mrak-
Copy link

mrak- commented Sep 7, 2023

Thanks to Vasily @le0pard.
Yes, it works manually, and so it works from gcloud cli and like a YAML file, but I need it that it would work from the terraform since my entire infrastructure is automated by the terraform. I would not want to perform part of the infra manually.
Sad my soul (((

@le0pard
Copy link
Author

le0pard commented Sep 7, 2023

@mrak- that is why this issue exists

@skadecl
Copy link

skadecl commented Sep 9, 2023

@le0pard have you been able to find a way to make this work?

@le0pard
Copy link
Author

le0pard commented Sep 9, 2023

@skadecl I already wrote how I resolved this issue in this comment - #15568 (comment)

No solution for terraform right now

@skadecl
Copy link

skadecl commented Sep 9, 2023

@skadecl I already wrote how I resolved this issue in this comment - #15568 (comment)

No solution for terraform right now

so for the yaml approach, are you deploying the cloud run instance entirely through gcloud cli or are you deploying it with terraform and then setting the vpc egress setting with gcloud?

@le0pard
Copy link
Author

le0pard commented Sep 9, 2023

@skadecl terraform don't manage cloud run at all, because if option will be change by yml, terraform cloud run resource will be broken and terraform will fail to apply cloud run. So gcloud cli with yml files manage cloud run resources

@skadecl
Copy link

skadecl commented Sep 9, 2023

@melinath @roaks3 any idea of when this could be supported?

@melinath
Copy link
Collaborator

If the API supports setting up direct egress with a VPC network, then it should be possible to support in Terraform. That would be the easiest fix. However, there is no specific timeline at the moment.

@yanweiguo
Copy link
Contributor

Direct VPC egress support was just added to Cloud Run v2 API. The public document hasn't been updated yet.

Terraform support is WIP.

@mrak-
Copy link

mrak- commented Sep 28, 2023

@github-actions
Copy link

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 29, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.