Upcoming Breaking Changes to Front Door / CDN's

[sebader]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

A few days ago I got his mail from Azure:

Before you delete Azure Front Door or Azure Content Delivery Network resources, remove their endpoint CNAME records from DNS
You're receiving this email because you use Azure Front Door or Content Delivery Network.
On 9 April 2021, we're updating Azure Front Door and Content Delivery Network to help prevent dangling DNS entries and the security risks they create. At that time, we'll start requiring the removal of canonical name (CNAME) records for Azure Front Door and Content Delivery Network resource endpoints from DNS before the resources can be deleted.
To delete Azure Front Door or Content Delivery Network resources, you must first remove the resource endpoint CNAME records from DNS starting on 9 April 2021.

I see a potential issue that could affect the terraform provider with the snippet below that I have been using so far (see below)

New or Affected Resource(s)

• azurerm_dns_cname_record
• azurerm_frontdoor

Potential Terraform Configuration

data "azurerm_dns_zone" "customdomain" {
  name                = var.custom_domain_name
  resource_group_name = var.dns_zone_resourcegroup_name
}

resource "azurerm_dns_cname_record" "app_subdomain" {
  name                = "${var.prefix}-app"
  zone_name           = data.azurerm_dns_zone.customdomain.name
  resource_group_name = var.dns_zone_resourcegroup_name
  ttl                 = 3600
  record              = "${var.prefix}frontdoor.azurefd.net"
}

resource "azurerm_frontdoor" "main" {
  name                             = "${var.prefix}frontdoor"
  resource_group_name   = azurerm_resource_group.main.name

  frontend_endpoint {
    name                     = "CustomDomainFrontendEndpoint"
    host_name                = azurerm_dns_cname_record.app_subdomain.fqdn
  }
}

This will obviously create a dependency graph from FrontDoor on the CNAME when you create the resources. But destroy would usually work the other way round - which will probably break things with the newly announced behavior.

I have no idea about potential workarounds but thought it's a good idea to at least raise awareness.

[tombuildsstuff]

hey @sebader

Thanks for opening this issue.

Unfortunately given the extremely short notice from the API team here and the inability to test this before it happens, to be honest it's hard for us to confirm the changes we're going to need to make to Terraform to handle this until after the breaking behavioural change in the API exists (since they may need to change, as happened with the Key Vault Soft Delete changes over Christmas/New Years).

Whilst we appreciate the heads up here (and have been tracking this ourselves. I'll pin this issue so others can find it) - as mentioned above I don't think we're going to be able to work around this before it happens, but it's likely this is going to necessitate a couple of breaking changes to both FrontDoor and (potentially) the CDN resources - once that happens (presumably Friday PST) we'll take a look into working around it.

@JeffreyRichter is there a reason this breaking change is being made on a Friday to an existing API version with several days notice, rather than in a new API version with a proper deprecation period?

Thanks!

[sebader]

thanks Tom! yeah I figured as well that it will be hard to assess actual impact before the change is implemented but I thought raising this here in advance can't hurt :)

[JeffreyRichter]

This is the first I'm hearing of the breaking change - thanks for bring it to my attention.
The Front Door should not be making this breaking change. If they must, then customers should get 3 years notice before being forced to update their code. @sebader, please follow up with me over MS email; thanks.

[sebader]

Ok, first result of my investigation: You can disable the behavior on a subscription level by registering the feature flag:

PS C:\Users\sebader> az feature list --query "[?name == 'Microsoft.Network/BypassCnameCheckForCustomDomainDeletion']"
[
  {
    "id": "/subscriptions/****-****-****-beed-f3baedfad7eb/providers/Microsoft.Features/providers/Microsoft.Network/features/BypassCnameCheckForCustomDomainDeletion",
    "name": "Microsoft.Network/BypassCnameCheckForCustomDomainDeletion",
    "properties": {
      "state": "NotRegistered"
    },
    "type": "Microsoft.Features/providers/features"
  }
]

Register like this

az feature register --namespace Microsoft.Network --name BypassCnameCheckForCustomDomainDeletion

Not saying that this is a solution but maybe at least a workaround

[ksmigiel]

@sebader thank you for that! I don't know how did you found this but it works.

[romeomorcia-wipro]

Hello!
I hope this will help someone that's struggling like me.

PS: I use modules and terragrunt for deploying my infra. CDN and DNS modules are separated and therefore the execution proceed in two different moments, causing the issue below because of the latest change of microsoft!

I think that this issue is related mostly to azurerm provider because it is not allowing me to delete CDN endpoints IF that endpoint has one or more DNS records connected.
This happens because my code works using multiple modules and all of them are connected through outputs.

Before, i was able to delete CDN endpoints without deleting DNS records, so after the module about CDN Endpoints completes, my DNS records module starts.

There are two workarounds for now:

• Create one module with all resources inside instead of keeping them separated in two modules
• DELETE manually or with a script the DNS records before running the terraform/terragrunt code.

So below you can find the error i was getting while i was trying to destroy JUST the azurerm_cdn_endpoint by having still up the DNS records.

tf version: Terraform v0.12.29
azurerm version: 2.55.0

azurerm_cdn_endpoint.cdn-endpt["something_deployed"]: Destroying... [id=/subscriptions/*************/endpoints/something-deployed]

Error: rpc error: code = Unavailable desc = transport is closing


panic: runtime error: invalid memory address or nil pointer dereference
2021-04-14T13:30:46.399+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5: [signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0x4e44e87]
2021-04-14T13:30:46.399+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5: 
2021-04-14T13:30:46.399+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5: goroutine 27 [running]:
2021-04-14T13:30:46.399+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5: github.com/terraform-providers/terraform-provider-azurerm/azurerm/internal/services/cdn.resourceCdnEndpointDelete(0xc0004d2d90, 0x5d59ea0, 0xc001b8c300, 0x0, 0x0)
2021-04-14T13:30:46.399+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/azurerm/internal/services/cdn/cdn_endpoint_resource.go:514 +0x247
2021-04-14T13:30:46.399+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5: github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Resource).Apply(0xc00094a990, 0xc001454f00, 0xc00190a040, 0x5d59ea0, 0xc001b8c300, 0x100df01, 0x20, 0x6110020)
2021-04-14T13:30:46.400+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/helper/schema/resource.go:283 +0x4a6
2021-04-14T13:30:46.400+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5: github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Provider).Apply(0xc00030fe00, 0xc001607a38, 0xc001454f00, 0xc00190a040, 0x9e92ee0, 0xc001454f00, 0x0)
2021-04-14T13:30:46.400+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/helper/schema/provider.go:294 +0x99
2021-04-14T13:30:46.400+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5: github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).ApplyResourceChange(0xc000307460, 0x6be2750, 0xc0014b41b0, 0xc0004d2070, 0xc000307460, 0xc0014b41b0, 0xc001a5dba0)
2021-04-14T13:30:46.400+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin/grpc_provider.go:885 +0x8a5
2021-04-14T13:30:46.400+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5: github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_ApplyResourceChange_Handler(0x622ede0, 0xc000307460, 0x6be2750, 0xc0014b41b0, 0xc0014c2060, 0x0, 0x6be2750, 0xc0014b41b0, 0xc0014c8c00, 0xaad)
2021-04-14T13:30:46.400+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5/tfplugin5.pb.go:3305 +0x214
2021-04-14T13:30:46.400+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5: google.golang.org/grpc.(*Server).processUnaryRPC(0xc00022ee00, 0x6c25618, 0xc0006c2c00, 0xc000312700, 0xc0005829f0, 0x9e50720, 0x0, 0x0, 0x0)
2021-04-14T13:30:46.400+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:1180 +0x52b
2021-04-14T13:30:46.400+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5: google.golang.org/grpc.(*Server).handleStream(0xc00022ee00, 0x6c25618, 0xc0006c2c00, 0xc000312700, 0x0)
2021-04-14T13:30:46.400+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:1503 +0xd0c
2021-04-14T13:30:46.400+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5: google.golang.org/grpc.(*Server).serveStreams.func1.2(0xc00129c1a0, 0xc00022ee00, 0x6c25618, 0xc0006c2c00, 0xc000312700)
2021-04-14T13:30:46.400+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:843 +0xab
2021-04-14T13:30:46.400+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5: created by google.golang.org/grpc.(*Server).serveStreams.func1
2021-04-14T13:30:46.400+0200 [DEBUG] plugin.terraform-provider-azurerm_v2.55.0_x5:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:841 +0x1fd
2021-04-14T13:30:46.405+0200 [WARN]  plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2021-04-14T13:30:46.406+0200 [DEBUG] plugin: plugin process exited: path=/Users/*****/plugins/darwin_amd64/terraform-provider-azurerm_v2.55.0_x5 pid=48727 error="exit status 2"
2021/04/14 13:30:46 [DEBUG] azurerm_cdn_endpoint.cdn-endpt["something_deployed"]: apply errored, but we're indicating that via the Error pointer rather than returning it: rpc error: code = Unavailable desc = transport is closing
2021/04/14 13:30:46 [TRACE] <root>: eval: *terraform.EvalWriteState
2021/04/14 13:30:46 [TRACE] EvalWriteState: writing current state object for azurerm_cdn_endpoint.cdn-endpt["something_deployed"]
2021/04/14 13:30:46 [TRACE] <root>: eval: *terraform.EvalApplyPost
2021/04/14 13:30:46 [ERROR] <root>: eval: *terraform.EvalApplyPost, err: rpc error: code = Unavailable desc = transport is closing
2021/04/14 13:30:46 [ERROR] <root>: eval: *terraform.EvalSequence, err: rpc error: code = Unavailable desc = transport is closing
2021/04/14 13:30:46 [ERROR] <root>: eval: *terraform.EvalOpFilter, err: rpc error: code = Unavailable desc = transport is closing
2021/04/14 13:30:46 [TRACE] [walkApply] Exiting eval tree: azurerm_cdn_endpoint.cdn-endpt["something_deployed"] (destroy)
2021/04/14 13:30:46 [TRACE] vertex "azurerm_cdn_endpoint.cdn-endpt[\"something_deployed\"] (destroy)": visit complete
2021/04/14 13:30:46 [TRACE] dag/walk: upstream of "provider.azurerm (close)" errored, so skipping
2021/04/14 13:30:46 [TRACE] dag/walk: upstream of "output.cdn_details" errored, so skipping
2021/04/14 13:30:46 [TRACE] dag/walk: upstream of "meta.count-boundary (EachMode fixup)" errored, so skipping
2021/04/14 13:30:46 [TRACE] dag/walk: upstream of "root" errored, so skipping
2021-04-14T13:30:46.545+0200 [DEBUG] plugin: plugin exited

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

[timja]

@sebader this only appears to cover front door? is there another feature for CDN?

[katbyte]

closing as internally discussed @WodansSon

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.