vpc_endpoint creation should enable Private DNS only after endpoint creation

[craiglink]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.11
provider.aws: version = "~> 1.55"

Affected Resource(s)

• resource aws_vpc_endpoint

Terraform Configuration Files

resource "aws_vpc_endpoint" "ec2" {
  vpc_id            = "${aws_vpc.main.id}"
  service_name      = "com.amazonaws.us-west-2.ec2"
  vpc_endpoint_type = "Interface"

  security_group_ids = [
    "${aws_security_group.sg1.id}",
  ]

  private_dns_enabled = true
}

Behavior

When the resource creates the vpc endpoint and enables private DNS, it creates short service outage because the DNS record becomes available before the endpoint is actually available. Because terraform is declarative, there is not a way to define a two-stage object creation, thus this behavior needs to be fixed in the provider or AWS SDK or service ( a feature request has been openned with AWS too )

To solve this via terraform, the provider could create the vpc endpoint without the Private DNS enable and then upon successful creation of the endpoint, modify it to enable Private DNS ( provided private_dns_enabled was set to true )

[ewbankkit]

@craiglink Terraform will wait until the VPC endpoint has available status before declaring the resource successfully created. Is there any way you can avoid using the newly resolving DNS record until the Terraform resource is created?

[rmlsun]

@ewbankkit It makes more sense to me to break out this private dns enablement attribute into a separate terraform resource. Below is my reasoning. (use VPCE for VPC endpoint for brevity)

It's more than the service consumer's need to wait for the available endpoint status. Rather, it's about one of Terraform's key values: auto detection and management of the order/sequence of resource provisioning with dependency among multiple discrete terraform resources.

Due the lifecycle of AWS interface VPCE, an interface VPCE initially will be always in a pending state. If the endpoint service provider chooses to not turn on auto-acceptance of interface VPCE connection request from a service consumer, an extra accept request action is needed before the interface VPCE state can transition to available.

Now this extra accept request action could be automated via terraform as well (and in our case, the vendor does have custom terraform provider for us (as service consumer) to trigger this accept request action with proper authN/Z).

So for the whole flow to work in terraform with the proper dependency chain, there needs to be 3 discrete terraform resource:
1). interface VPCE provision
2). accept endpoint connection request (via service provider terraform provider and in turn their API, probably with authN/Z)
3). interface VPCE private DNS enablement

With the current VPCE resource design bundling 1) and 3) into one single TF resource, it's impossible to properly automate the whole flow via terraform (unless the service provider turn on VPCE auto-accept).

[sqlbot]

A much more severe manifestation of this issue arises when attempting to use aws_vpc_endpoint to create an endpoint for a non-AWS service created elsewhere with aws_vpc_endpoint_service using acceptance_required = true. It's not possible.

| Error: creating EC2 VPC Endpoint (com.amazonaws.vpce.us-gov-east-1.vpce-svc-xxxx): InvalidParameter: Private DNS can only be enabled after the endpoint connection is accepted by the owner of com.amazonaws.vpce.us-gov-east-1.vpce-svc-xxxx.
│       status code: 400, request id: 8f53996a-6968-4164-9003-xxxxxxxxxxxx
│
│   with aws_vpc_endpoint.this,
│   on main.tf line 28, in resource "aws_vpc_endpoint" "this":
│   28: resource "aws_vpc_endpoint" "this" {

The provider is attempting an action that the underlying API will always deny on a service that requires acceptance. There is no opportunity to use aws_vpc_endpoint_connection_accepter because the Terraform aws_vpc_endpoint resource has not been and cannot be created.

The only workaround appears to be to set private_dns_enabled = false inside aws_vpc_endpoint, then terraform apply, then change to private_dns_enabled = true and terraform apply again.

This is definitely not viable. The provider almost certainly needs to wait for available before ever attempting to set the value to true.

Terraform v1.2.1 on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v4.28.0

[github-actions]

Warning

This issue has been closed, meaning that any additional comments are hard for our team to see. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.

[github-actions]

This functionality has been released in v5.51.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.