resource/aws_acm_certificate: Error with private_key argument when updating an imported certificate

[tsacha]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.11

• provider.aws: version = "~> 1.57"
• provider.local: version = "~> 1.1"

Affected Resource(s)

• aws_acm_certificate

Terraform Configuration Files

data "local_file" "public_key" {
  filename = "/foo/${var.domain_name}/server.crt"
}

data "local_file" "private_key" {
  filename = "/fool/${var.domain_name}/server.key"
}

resource "aws_acm_certificate" "cert" {
  private_key       = "${data.local_file.private_key.content}"
  certificate_body  = "${element(split("-----END CERTIFICATE-----", data.local_file.public_key.content), 0)}-----END CERTIFICATE-----"
  certificate_chain = "${data.local_file.public_key.content}"
  provider          = "aws.virginia"

  lifecycle {
    create_before_destroy = true
  }

  tags {
    Name = "${var.domain_name}"
    Terraform = true
  }
}

Debug Output

* aws_acm_certificate.cert: Error updating certificate: ValidationException: The private key is not supported.
	status code: 400, request id: xxxx

Trace output: https://gist.github.com/tsacha/2ceba19db9b04f459cfbd8fb8b166b6f

I tried to add a print here

log.Printf("Private key: %s", d.Get("private_key"))

Instead of the file content, resource ID is printed. My Terraform knowledge is not enough to continue.

Panic Output

Expected Behavior

Non-Amazon issued certificate is updated.

Actual Behavior

Certificates are not modified on Amazon.

Steps to Reproduce

1. Import with Terraform a Let's Encrypt certificate
2. Update the certificate
3. Try to import the updated certificate with Terraform

[bflad]

Hi @tsacha 👋 Sorry for the strange behavior here.

It appears there might be something awry outside the aws_acm_certificate resource in your scenario. For reference, we do verify aws_acm_certificate updates with the TestAccAWSAcmCertificate_imported_DomainName acceptance test, which is consistently passing and applies the following plan:

UPDATE: aws_acm_certificate.cert
  certificate_body: "4b030216f1f3e206894c24d8963055d3738f3d53" => "<computed>"
  private_key:      "<sensitive>" => "<sensitive>" (attribute changed)
DESTROY: tls_private_key.example
CREATE: tls_private_key.example2
  algorithm:          "" => "RSA" (forces new resource)
  ecdsa_curve:        "" => "P224" (forces new resource)
  private_key_pem:    "" => "<computed>"
  public_key_openssh: "" => "<computed>"
  public_key_pem:     "" => "<computed>"
  rsa_bits:           "" => "2048" (forces new resource)
DESTROY: tls_self_signed_cert.example
CREATE: tls_self_signed_cert.example2
  allowed_uses.#:         "" => "3" (forces new resource)
  allowed_uses.0:         "" => "key_encipherment" (forces new resource)
  allowed_uses.1:         "" => "digital_signature" (forces new resource)
  allowed_uses.2:         "" => "server_auth" (forces new resource)
  cert_pem:               "" => "<computed>"
  early_renewal_hours:    "" => "0" (forces new resource)
  key_algorithm:          "" => "RSA" (forces new resource)
  private_key_pem:        "" => "<computed>" (forces new resource)
  subject.#:              "" => "1" (forces new resource)
  subject.0.common_name:  "" => "example2.com"
  subject.0.organization: "" => "ACME Examples, Inc"
  validity_end_time:      "" => "<computed>"
  validity_period_hours:  "" => "12" (forces new resource)
  validity_start_time:    "" => "<computed>"

Making the following ImportCertificate API call successfully:

2019/01/25 04:33:10 [DEBUG] ACM Certificate Import: {
  Certificate: <binary> len 1306,
  CertificateArn: "arn:aws:acm:us-west-2:--OMITTED--:certificate/42461506-94be-4807-ae69-c8bc0b0373be",
  PrivateKey: <binary> len 1675
}
2019/01/25 04:33:10 [DEBUG] [aws-sdk-go] DEBUG: Request acm/ImportCertificate Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: acm.us-west-2.amazonaws.com
User-Agent: aws-sdk-go/1.16.19 (go1.11.4; linux; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.11.9-beta1
Content-Length: 4117
Authorization: AWS4-HMAC-SHA256 Credential=AKIAIXP7QHO656Q4VL7Q/20190125/us-west-2/acm/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-target, Signature=e54d97f26e349a9fde920a1a12666c1ff4ad1cda11c540fbce1ae3e1d816e70b
Content-Type: application/x-amz-json-1.1
X-Amz-Date: 20190125T043310Z
X-Amz-Target: CertificateManager.ImportCertificate
Accept-Encoding: gzip

{"Certificate":"LS0tLS1...--OMITTED--...LS0tCg==","CertificateArn":"arn:aws:acm:us-west-2:--OMITTED--:certificate/42461506-94be-4807-ae69-c8bc0b0373be","PrivateKey":"LS0t...--OMITTED--...S0tCg=="}
-----------------------------------------------------
2019/01/25 04:33:11 [DEBUG] [aws-sdk-go] DEBUG: Response acm/ImportCertificate Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK

The PrivateKey: <binary> len 40 in your output does certainly seem suspect. You will need to ensure the file path is correct for reading the file (is the extra l in the key data source filename intentional?) and that Terraform is correctly referenced between the data source and resource. If all else fails, you can try hard coding the private key directly into your configuration temporarily just to rule out strangeness with the data source.

Let us know!

[tsacha]

Hi @bflad, thanks for your reply.

During the certificate renewal, private key is unmodified. Maybe my scenario is not the same than TestAccAWSAcmCertificate_imported_DomainName where private_key has attribute changed?

I don't think I've made mistakes with my data sources: the first import is working correctly. The extra l is a typo when I removed sensitive file path, sorry.

Thanks!

[leva0887]

Same issue for me. Any updates here?

[sunil238174]

I faced similar issue when I tried updating my existing certificate in aws cert manager.Its not allowing update certificate.

aws_acm_certificate.cert: Error updating certificate: ValidationException: com.amazonaws.pki.acm.exceptions.external.ValidationException: Could not validate the certificate with the certificate chain.
status code: 400, request id:

[uuqq]

I've been having that same updating certificate error where it couldn't validate the certificate using the provided certificate chain so now I'm tainting the resource before running the terraform apply... Can't seem to find any other way around this at the moment. Note: I am using Terraform 0.11.14, I don't know if upgrading to 0.12 will fix this.

[G-Rath]

I've had this in v0.12.9, w/ the following module:

resource "tls_private_key" "key" {
  algorithm = "RSA"
}

resource "tls_self_signed_cert" "cert" {
  key_algorithm   = "RSA"
  private_key_pem = tls_private_key.key.private_key_pem

  subject {
    common_name  = var.common_name
  }

  validity_period_hours = 120
  early_renewal_hours = 10

  allowed_uses = [
    "key_encipherment",
    "digital_signature",
    "server_auth",
  ]
}

resource "aws_acm_certificate" "cert" {
  private_key      = tls_private_key.key.private_key_pem
  certificate_body = tls_self_signed_cert.cert.cert_pem
}

It works fine for creation, but if I modify the early_renewal_hours or validity_period_hours properties, applying fails w/ the error described by this issue.

[cxfcxf]

i think the problem is StateFunc: normalizeCert
if one of those certs does not change, the function call d.Get() returns a SHA1 hashed cert (40 digits) from state file since its normalized when it get stored in state file section
the reason of fetching the SHA1 is due to in planning it does not detect changes on the cert

a better solution will be using base64 encoding/decoding on the cert, so it can be convert back when needed instead of take a one direction hash SHA1.

this also applies to resource_aws_iam_server_certificate.go, we dont use it, so i dont know

[bflad]

Hi folks 👋 The fix for this, saving the full attribute values in the Terraform state instead of hashed values, has been merged and will release with version 3.0.0 of the Terraform AWS Provider, likely in two weeks.

The version 3 upgrade guide will outline some details about a one-time terraform apply behavior that will occur after upgrade for the aws_acm_certificate resource:

Previously when the certificate_body, certificate_chain, and private_key arguments were stored in state, they were stored as a hash of the actual value. This prevented Terraform from properly updating the resource when necessary and the hashing has been removed. The Terraform AWS Provider will show an update to these arguments on the first apply after upgrading to version 3.0.0, which is fixing the Terraform state to remove the hash. Since the private_key attribute is marked as sensitive, the values in the update will not be visible in the Terraform output. If the non-hashed values have not changed, then no update is occurring other than the Terraform state update. If these arguments are the only updates and they all match the hash removal, the apply will occur without submitting API calls.

If you have trouble after upgrading to version 3.0.0 of the Terraform AWS Provider, please create a new issue and we will take a look. Thanks so much and apologies for the frustrating behavior in the meantime.

[ghost]

This has been released in version 3.0.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!