Add aws_acm_certificate resource with automated verification with Route53

[pdecat]

Since 2017/11/22, AWS ACM now supports request verification via DNS records in addition to email verification:

https://aws.amazon.com/fr/about-aws/whats-new/2017/11/aws-certificate-manager-easier-certificate-validation-using-dns/
https://aws.amazon.com/certificate-manager/faqs/#dns_validation
http://docs.aws.amazon.com/acm/latest/userguide/gs-acm-validate-dns.html

This could be used with supported DNS resources from AWS provider (and probably others) to automatically request and verify ACM certificates.

Terraform Version

Terraform v0.11.0

• provider.aws v1.3.1

Affected Resource(s)

• aws_acm_certificate

Terraform Configuration Files

An example of what an automated ACM request and validation with Route53 would look like configuration wise:

resource "aws_route53_zone" "example" {
  name = "example.com"
}

# NOTE: this resource does not exist yet
resource "aws_acm_certificate" "acm_tf" {
  domain = "tf.example.com"

  subject_alternative_names = [
     "tf1.example.com",
     "tf2.example.com",
  ]
}

resource "aws_route53_record" "acm_tf_verification" {
  zone_id = "${aws_route53_zone.example.zone_id}"
  name    = "${aws_acm_certificate.acm_tf.dns_cname_name}.example.com"
  type    = "CNAME"
  ttl     = "30"

  records = [
    "${aws_acm_certificate.acm_tf.dns_cname_value}",
  ]
}

[pdecat]

On second thought, maybe the new aws_acm_certificate resource would have to do the DNS record creation by itself in order to be able to wait for validation completion (i.e. until the ACM cert is in ISSUED status).

[tomelliff]

The ACM certificate resource would only be able to create the DNS record itself if you were using Route53. It would massively simplify the Terraform control flow though and there's precedent for it in a couple of other resources where IAM service roles are created automatically if they aren't already there. Of course that would only be possible if you are using R53 for your DNS but I'd guess that probably covers a large chunk of people who would like to use this resource.

There was some talk on the old issue about maybe having a wait_for_approval boolean that would mean the ACM cert request could at least be created with Terraform and approval could happen outside of that workflow.

[jleclanche]

It's also possible to use the cloudflare plugin if using cloudflare instead of route53, but that's crossing plugin boundaries.

The other thing though is that according to the amazon documentation, the verification domain name doesn't change for a domain name; which means once the verification txt is issued, as long as it remains live it doesn't need to be changed/updated, so the approval step could even be skipped in some cases.

[flosell]

@pdecat I like your original solution. I think we could solve the "waiting for certificate issued" problem by moving it to the data source.

The resource would make the RequestCertificate request and return information for the DNS validation. Any DNS resource could then add the DNS record. The data source would then wait for the certificate being issued:

resource "aws_route53_zone" "example" {
  name = "example.com"
}

# NOTE: this resource does not exist yet
resource "aws_acm_certificate" "acm_tf" {
  domain = "tf.example.com"

  subject_alternative_names = [
     "tf1.example.com",
     "tf2.example.com",
  ]
}

resource "aws_route53_record" "acm_tf_verification" {
  zone_id = "${aws_route53_zone.example.zone_id}"
  name    = "${aws_acm_certificate.acm_tf.dns_cname_name}.example.com"
  type    = "CNAME"
  ttl     = "30"

  records = [
    "${aws_acm_certificate.acm_tf.dns_cname_value}",
  ]
}

data "aws_acm_certificate" "acm_tf" {
  statuses = ["ISSUED"]
  # NOTE: this does not exist yet but would make sense to have an explicit dependency
  # and reference the right certificate in case there's more than one for the same domain
  certificate_arn = "${aws_acm_certificate.acm_tf.arn}"

  # NOTE: this does not exist yet. Would cause the data source to poll until a timeout is reached or the certificate is in the correct status
  wait_for_certificate = true
}

# USAGE: use the arn from the datasource to make sure you are using an issued certificate
resource "aws_lb_listener" "front_end" {
  certificate_arn   = "${data.aws_acm_certificate.acm_tf}"

}

To me, this feels similar to what @vancluever proposed for the ACME provider.

This should work in most cases where certificate is used via the data source. I'm a bit unsure what happens if the certificate is not used in the same terraform project, i.e. there's no immediate dependency that would block the apply.

[apparentlymart]

Hi everyone! Thanks for the great discussion here.

Following the model proposed for ACME seems like a good idea, to separate the three steps:

• Request a certificate and get a challenge
• Act on that challenge using some other resource, possibly in an entirely separate provider
• Obtain the result of completing the challenge

Using polling and a timeout for the completion, rather than having a resource that explicitly declares the challenge as completed, is not ideal but indeed seems like the best thing that can be done with the design of this API, and should be okay as long as ACM is able to respond to the completion fast enough that Terraform won't appear to hang for a long time. Ideally we'd want to also be able to detect a failure quickly and return an explicit error message, rather than simply timing out, but perhaps that isn't possible.

The Terraform Team at HashiCorp doesn't have any immediate plan to work on this due to our focus being elsewhere, but we'd be happy to review a PR if someone in the community has the time and motivation to work on it. Alternatively, feel free to upvote the original issue (not this comment!) if you'd find this feature useful since we use issue votes as one of the inputs into our prioritization process.

[flosell]

Gave implementing this a shot. Here's the current state: #2801.
I'd be happy for feedback.

[xenoterracide]

any ideas which version this will be available in? this is my most wanted feature from terraform (my most wanted from aws is not to require a challenge if you're using route53, and the domain exists)

[flosell]

@xenoterracide PR #2813 implements this feature, and is waiting for review. Unless there are fundamental problems I'll probably be able to address feedback quickly to get this ready to merge.

[bflad]

As a heads up for anyone still watching here, I'm about to submit the first round of maintainer feedback on #2813. Feel free to follow along there! 😄

[bflad]

Hi everyone! Today is your lucky day as we have just merged in #2813 into master, which provides the two aforementioned new resources: aws_acm_certificate and aws_acm_certificate_validation. They will be released along with their documentation in v1.9.0 of the AWS provider, expecting to ship at the end of this week.

HUGE shoutout to @flosell, @pdecat, @apparentlymart, and the others involved with making this a reality. 😂

[bflad]

This has been released in terraform-provider-aws version 1.9.0. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[xenoterracide]

have problems creating this with a cloudfront distribution at the same time, worked on a later run, my guess is need to wait for a read back

Error: Error applying plan:

1 error(s) occurred:

* module.ArtifactsCloudfront.aws_cloudfront_distribution.Distribution: 1 error(s) occurred:

* aws_cloudfront_distribution.Distribution: InvalidViewerCertificate: The specified SSL certificate doesn't exist, isn't in us-east-1 region, isn't valid, or doesn't include a valid certificate chain.
        status code: 400, request id: 554e13bd-21fe-11e8-ba3a-fd16cae728a3

here's the module code

variable "user_fqdn" {}
variable "origin_fqdn" {}
variable "origin_port" {}
variable "zone_id" {}
variable "root" {}


resource "aws_acm_certificate" "Cert" {
    domain_name = "${var.user_fqdn}"
    validation_method = "DNS"
    tags {
        env = "${terraform.env}"
    }
}

resource "aws_route53_record" "ValidationDNS" {
    name = "${aws_acm_certificate.Cert.domain_validation_options.0.resource_record_name}"
    type = "${aws_acm_certificate.Cert.domain_validation_options.0.resource_record_type}"
    zone_id = "${var.zone_id}"
    records = ["${aws_acm_certificate.Cert.domain_validation_options.0.resource_record_value}"]
    ttl = 60
}

resource "aws_acm_certificate_validation" "CertValidation" {
    certificate_arn = "${aws_acm_certificate.Cert.arn}"
    validation_record_fqdns = ["${aws_route53_record.ValidationDNS.fqdn}"]
}

resource "aws_route53_record" "DomainName" {
    zone_id = "${var.zone_id}"
    name = "${var.user_fqdn}"
    type = "CNAME"
    ttl = "300"
    records = [
        "${aws_cloudfront_distribution.Distribution.domain_name}"]
}

resource "aws_cloudfront_distribution" "Distribution" {
    aliases = ["${var.user_fqdn}"]
    origin {
        origin_id = "${var.origin_fqdn}"
        domain_name = "${var.origin_fqdn}"
        custom_origin_config {
            http_port = "${var.origin_port}",
            https_port = 443,
            origin_protocol_policy = "http-only"
            origin_ssl_protocols = [
                "TLSv1.2"]
        }
    }

    enabled = true
    is_ipv6_enabled = true
    default_root_object = "${var.root}"

    default_cache_behavior {
        allowed_methods = [
            "GET",
            "HEAD",
            "OPTIONS",
            "PUT",
            "POST",
            "PATCH",
            "DELETE"]
        cached_methods = [
            "GET",
            "HEAD",
            "OPTIONS"]

        forwarded_values {
            cookies {
                forward = "all"
            }
            headers = [
                "Accept",
                "Authorization",
                "Host",
                "X-Forwarded-Proto",
                "CloudFront-Forwarded-Proto",
            ]

            query_string = true
        }
        default_ttl = 3600
        max_ttl = 86400
        min_ttl = 60
        target_origin_id = "${var.origin_fqdn}"
        viewer_protocol_policy = "https-only"
        compress = true
    }

    price_class = "PriceClass_100"

    restrictions {
        geo_restriction {
            restriction_type = "whitelist"
            locations = ["US"]
        }
    }

    viewer_certificate {
        acm_certificate_arn = "${aws_acm_certificate.Cert.arn}"
        ssl_support_method = "sni-only"
        minimum_protocol_version = "TLSv1.1_2016"
    }
    tags {
        env = "${terraform.env}"
    }
}

[bflad]

@xenoterracide you might be able to fix your situation by referring to the aws_acm_certificate_validation instead of a new aws_acm_certificate (CloudFront likely cannot use certificates that are not in the ISSUED status yet.

I'd try replacing this:

acm_certificate_arn = "${aws_acm_certificate.Cert.arn}"

With the below, which will ensure the certificate is fully ISSUED before trying to create the CloudFront distribution:

acm_certificate_arn = "${aws_acm_certificate_validation.CertValidation.certificate_arn}"

[oarmstrong]

What @bflad mentioned above is exactly what works for me. I did it using the aws_acm_certificate resource by mistake on my first implementation but using aws_acm_certificate_validation works for me now.

[xenoterracide]

sorry for using this for what's essentially questions

resource "aws_route53_record" "ValidationDNS0" {
    name = "${aws_acm_certificate.Cert.domain_validation_options.0.resource_record_name}"
    type = "${aws_acm_certificate.Cert.domain_validation_options.0.resource_record_type}"
    zone_id = "${var.route53_zone_id}"
    records = ["${aws_acm_certificate.Cert.domain_validation_options.0.resource_record_value}"]
    ttl = 300
}

resource "aws_route53_record" "ValidationDNS1" {
    name = "${aws_acm_certificate.Cert.domain_validation_options.1.resource_record_name}"
    type = "${aws_acm_certificate.Cert.domain_validation_options.1.resource_record_type}"
    zone_id = "${var.route53_zone_id}"
    records = ["${aws_acm_certificate.Cert.domain_validation_options.1.resource_record_value}"]
    ttl = 300
}

resource "aws_acm_certificate_validation" "CertValidation" {
    certificate_arn = "${aws_acm_certificate.Cert.arn}"
    validation_record_fqdns = ["${aws_route53_record.ValidationDNS0.fqdn}", "${aws_route53_record.ValidationDNS1.fqdn}"]
}

is there any way to dry this up? I tried to use * instead of 0 and 1, but got additional errors... if so perhaps the docs could be updated to use that, as what's good for many should still be good for 1? https://www.terraform.io/docs/providers/aws/r/acm_certificate_validation.html

[oarmstrong]

@xenoterracide this works for me:

variable "domain" {
  default = "example.com"
}

variable "san_domains" {
  default = [
    "www.example.com",
  ]
}

resource "aws_acm_certificate" "example" {
  domain_name               = "${var.domain}"
  validation_method         = "DNS"
  subject_alternative_names = "${var.san_domains}"
}

resource "aws_route53_record" "example" {
  count   = "${length(var.san_domains) + 1}"
  zone_id = "${aws_route53_zone.example.id}"
  name    = "${lookup(aws_acm_certificate.example.domain_validation_options[count.index], "resource_record_name")}"
  type    = "${lookup(aws_acm_certificate.example.domain_validation_options[count.index], "resource_record_type")}"
  records = ["${lookup(aws_acm_certificate.example.domain_validation_options[count.index], "resource_record_value")}"]
  ttl     = 60
}

[xenoterracide]

thanks, made some tweaks, but this looks like it works

resource "aws_acm_certificate" "Cert" {
    domain_name = "${var.alt_fqdn}"
    subject_alternative_names = ["${var.user_fqdn}"]
    validation_method = "DNS"
    tags {
        env = "${terraform.env}"
    }
}

resource "aws_route53_record" "ValidationDNS" {
    count = "${length(aws_acm_certificate.Cert.domain_validation_options)}"
    name = "${lookup( aws_acm_certificate.Cert.domain_validation_options[count.index], "resource_record_name")}"
    type = "${lookup( aws_acm_certificate.Cert.domain_validation_options[count.index], "resource_record_type")}"
    zone_id = "${var.route53_zone_id}"
    records = ["${lookup( aws_acm_certificate.Cert.domain_validation_options[count.index], "resource_record_value") }"]
    ttl = 300
}

resource "aws_acm_certificate_validation" "CertValidation" {
    certificate_arn = "${aws_acm_certificate.Cert.arn}"
    validation_record_fqdns = ["${aws_route53_record.ValidationDNS.*.fqdn}" ]
}

due to the fact that I think this should work independently of actual count on the cert I do think this should be shown in the docs, instead of simply how to do it with 1 domain.

I was changing the domain names and encountered this (after previous discussion on cloudfront)

* aws_acm_certificate.Cert: Error deleting certificate: ResourceInUseException: Certificate arn:aws:acm:us-east-1:927476265057:certificate/bbcd1035-c2b8-404a-a288-5cd966fbfc27 in account 927476265057 is in use.
	status code: 400, request id: 35f56fc1-23da-11e8-a9bb-ed94dd4ae19a

I worked around it by hand going in and changing the cert to use default cert, which allowed destruction/changes to proceed, but seems cludgy

[hammadzz]

resource "aws_acm_certificate" "cert" {
  domain_name               = "${var.domain}"
  validation_method         = "DNS"
  subject_alternative_names = ["${var.san_domains}"]

  tags {
    Name        = "${var.stage}"
    Stage       = "${var.stage}"
    Environment = "${var.environment}"
  }
}

resource "aws_route53_record" "cert_validation" {
  count   = "${length(var.san_domains) + 1}"
  name    = "${lookup(aws_acm_certificate.cert.domain_validation_options[count.index], "resource_record_name")}"
  type    = "${lookup(aws_acm_certificate.cert.domain_validation_options[count.index], "resource_record_type")}"
  zone_id = "${var.zone_id}"
  records = ["${lookup(aws_acm_certificate.cert.domain_validation_options[count.index], "resource_record_value")}"]
  ttl     = 60
}

resource "aws_acm_certificate_validation" "cert" {
  certificate_arn = "${aws_acm_certificate.cert.arn}"
  validation_record_fqdns = ["${aws_route53_record.cert_validation.*.fqdn}"]
}

output "cert_arn" {
  value = "${aws_acm_certificate_validation.cert.certificate_arn}"
}

It works in validating ACM but terraform thinks that something is wrong. I am using the following (my domain swapped for example.com):
var.domain = example.com
var.san_domains = ["www.test.example.com", "*.test.example.com"]

aws_acm_certificate_validation.cert: Certificate needs
[_c458895c0781030e5b109ae233e11a42.www.test.example.com
_f2b93324df31ae587e54a160aeb606fe.test.example.com
_f2b93324df31ae587e54a160aeb606fe.test.example.com] to be set but only
[_c458895c0781030e5b109ae233e11a42.www.test.example.com
_f2b93324df31ae587e54a160aeb606fe.test.example.com] 
was passed to validation_record_fqdns

Problem is with having a wildcard in there. In my past experience I had issues with just having sans_domains with *.example.com. Maybe I am doing my certs wrong.

[hammadzz]

So wildcard cert is a problem

Q. How does ACM construct CNAME records?

DNS CNAME records have two components: a name and a label. The name component of an ACM-generated CNAME is constructed from an underscore character (_) followed by a token, which is a unique string that is tied to your AWS account and your domain name. ACM prepends the underscore and token to your domain name to construct the name component. ACM constructs the label from an underscore character prepended to a different token which is also tied to your AWS account and your domain name. ACM prepends the underscore and token to a DNS domain name used by AWS for validations: acm-validations.aws. The following examples show the formatting of CNAMEs for www.example.com, subdomain.example.com, and *.example.com.

_TOKEN1.www.example.com CNAME _TOKEN2.acm-validations.aws
_TOKEN3.subdomain.example.com CNAME _TOKEN4.acm-validations.aws
_TOKEN5.example.com CNAME _TOKEN6.acm-validations.aws

Notice that ACM removes the wildcard label (*) when generating CNAME records for wildcard names. As a result, the CNAME record generated by ACM for a wildcard name (such as *.example.com) is the same record returned for the domain name without the wildcard label (example.com).

https://aws.amazon.com/certificate-manager/faqs/

[voroniys]

resource "aws_route53_record" "ValidationDNS" {
count = "${length(aws_acm_certificate.Cert.domain_validation_options)}"

I've tried the same and it does not work:

• aws_route53_record.cert_validation_dns_record: aws_route53_record.cert_validation_dns_record: value of 'count' cannot be computed

It does work if certificate is created beforehand, but if you'd like to do it in one run it does not.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!