Changing certificate validation records, triggers diff bug

[dekimsey]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.14
+ provider.aws v2.13.0
+ provider.azuread v0.3.1
+ provider.null v2.1.2
+ provider.random v2.1.2
+ provider.template v2.1.2
+ provider.tls v2.0.1

Your version of Terraform is out of date! The latest version
is 0.12.0. You can update by downloading from www.terraform.io/downloads.html

Affected Resource(s)

• aws_route53_record
• aws_acm_certificate_validation

Terraform Configuration Files

## request cert
resource "aws_acm_certificate" "elastichq" {
  domain_name       = "${aws_route53_record.elastichq.fqdn}"
  validation_method = "DNS"

  tags = {
    terraform = "true"
    namespace = "${local.unique_namespace}"
  }

  lifecycle {
    create_before_destroy = true
  }
}

## create TLS validation record
resource "aws_route53_record" "elastichq_cert_validation" {
  name    = "${aws_acm_certificate.elastichq.domain_validation_options.0.resource_record_name}"
  type    = "${aws_acm_certificate.elastichq.domain_validation_options.0.resource_record_type}"
  zone_id = "${aws_route53_zone.external.id}"
  records = ["${aws_acm_certificate.elastichq.domain_validation_options.0.resource_record_value}"]
  ttl     = 60
}

# Link the Cert and the Validation record together
resource "aws_acm_certificate_validation" "elastichq" {
  certificate_arn         = "${aws_acm_certificate.elastichq.arn}"
  validation_record_fqdns = ["${aws_route53_record.elastichq_cert_validation.fqdn}"]
}

Debug Output

Error: Error applying plan:

10 errors occurred:
	* aws_route53_record.elastichq_cert_validation: aws_route53_record.elastichq_cert_validation: diffs didn't match during apply. This is a bug with Terraform and should be reported as a GitHub Issue.

Please include the following information in your report:

    Terraform Version: 0.11.14
    Resource ID: aws_route53_record.elastichq_cert_validation
    Mismatch reason: attribute mismatch: records.3350658573
    Diff One (usually from plan): *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{"allow_overwrite":*terraform.ResourceAttrDiff{Old:"true", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "fqdn":*terraform.ResourceAttrDiff{Old:"_134dc8a2d086cda9175a815352660a5e.elastichq.apj.prod.example.cloud", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "name":*terraform.ResourceAttrDiff{Old:"_134dc8a2d086cda9175a815352660a5e.elastichq.apj.prod.example.cloud", New:"_134dc8a2d086cda9175a815352660a5e.elastichq.apj.prod.example.cloud", NewComputed:false, NewRemoved:false, NewExtra:"_134dc8a2d086cda9175a815352660a5e.elastichq.apj.prod.example.cloud.", RequiresNew:false, Sensitive:false, Type:0x0}, "records.#":*terraform.ResourceAttrDiff{Old:"1", New:"1", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "records.3350658573":*terraform.ResourceAttrDiff{Old:"_bdd0984a7f7674bd51dfe6bb4cef53cb.hkvuiqjoua.acm-validations.aws.", New:"_bdd0984a7f7674bd51dfe6bb4cef53cb.hkvuiqjoua.acm-validations.aws.", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ttl":*terraform.ResourceAttrDiff{Old:"60", New:"60", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "type":*terraform.ResourceAttrDiff{Old:"CNAME", New:"CNAME", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "zone_id":*terraform.ResourceAttrDiff{Old:"Z2QWUKJCLSN4AL", New:"${aws_route53_zone.external.id}", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, Meta:map[string]interface {}(nil)}
    Diff Two (usually from apply): *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{"allow_overwrite":*terraform.ResourceAttrDiff{Old:"", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "fqdn":*terraform.ResourceAttrDiff{Old:"", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "name":*terraform.ResourceAttrDiff{Old:"", New:"_935faa64a80554a1dc88a218ab41861d.elastichq.apj.example.cloud", NewComputed:false, NewRemoved:false, NewExtra:"_935faa64a80554a1dc88a218ab41861d.elastichq.apj.example.cloud.", RequiresNew:true, Sensitive:false, Type:0x0}, "records.#":*terraform.ResourceAttrDiff{Old:"", New:"1", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "records.2471609582":*terraform.ResourceAttrDiff{Old:"", New:"_1e2a8b44ac933d371ca3a9a6a08b2db4.ltfvzjuylp.acm-validations.aws.", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ttl":*terraform.ResourceAttrDiff{Old:"", New:"60", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "type":*terraform.ResourceAttrDiff{Old:"", New:"CNAME", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "zone_id":*terraform.ResourceAttrDiff{Old:"", New:"ZGCL5P4RQ250M", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, Meta:map[string]interface {}(nil)}

Also include as much context as you can about your config, state, and the steps you performed to trigger this error.

Panic Output

Expected Behavior

Run should replace certs correctly.

Actual Behavior

Crashes.

Steps to Reproduce

1. terraform apply
2. Change the aws_route53_record's zone
3. terraform apply # errors
4. terraform apply # completes successfully

Important Factoids

Crash occurs for all our certificates managed this way.

References

• #0000

[bflad]

Hi everyone 👋 Sorry you ran into this confusing error and thank you for reporting it. There have been a lot of relevant changes in the Terraform Core and Terraform AWS Provider codebases since this issue was originally reported so as a first step it might be best to try updating to recent versions of both and seeing if any of those updates may have resolved this issue.

Attempting to troubleshoot these types of errors is very difficult in Terraform 0.11 and earlier as those versions will report this problem (diffs didn't match during apply error) where a problematic configuration value exists or is referenced and not where the Terraform resource with the underlying problem actually exists. We typically will need much more information than the original error message suggests for filing the issue (e.g. debug logs). There are also frequent issues in the Terraform 0.11 error reporting where the error message is due to behavior outside the control of the Terraform resources themselves (e.g. lifecycle configurations).

In Terraform 0.12 and later, the "value did not match" type of error can now be found with the newer Provider produced inconsistent result after apply error, which better reports the source of the problem so the provider maintainers and community can more easily work towards a fix of the issue. Other similar errors that fell previously into diffs didn't match during apply errors, but outside problems within the Terraform provider are also better diagnosed before being displayed.

Since the majority of the Terraform code path and reporting structure of these errors has changed between Terraform 0.11 and 0.12, along with the difficult triage process in Terraform 0.11 and earlier, the maintainers prefer to close these older issues to continue troubleshooting and fixing based on Terraform 0.12's error reporting instead. I apologize for this unsatisfying closure of this particular version of the issue.

That said, if the configuration does happen to reproduce the newer Provider produced inconsistent result after apply error in Terraform 0.12 (which requires Terraform AWS Provider version 2.7.0 or later), please do not hesitate to file a new issue and complete all of the information requested in the Bug Report template so we can hopefully get to the root cause for fixing the issue. It is very likely, however, that the root cause of this issue is #8531, just that Terraform 0.11 reports it confusingly. We plan on fixing #8531 as part of the next major version upgrade of the Terraform AWS Provider, which can also be tracked in #13053. Thanks again for this bug report and sorry about the additional steps.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!