tls_cert_request with count forces new resource every time #8764

RichardKnop · 2016-09-10T11:30:44Z

Hi there,

I am building a CoreOS cluster with Terraform and I am creating a locally signed certificate for each cluster node. Something along the lines:

data "tls_cert_request" "server" {
  count = "${var.cluster_size}"
  ...
}

resource "tls_locally_signed_cert" "server" {
  count = "${var.cluster_size}"
  cert_request_pem = "${element(data.tls_cert_request.server.*.cert_request_pem, count.index)}"
  ...
}

Then I use the generated locally signed certificate as part of user data to spin up cluster nodes.

The problem is, Terraform forces the instances to be recreated every time without any changes.

It thinks the tls_cert_request has changed although it should not have.

Terraform Version

Terraform v0.7.4-dev (291f2985354454749e3e6da586c1ecd5d245c181)

Affected Resource(s)

tls_cert_request

Debug Output

See bellow. This happens every single time and forces new resources.

-/+ module.cluster.tls_locally_signed_cert.server
    allowed_uses.#:        "3" => "3"
    allowed_uses.0:        "signing" => "signing"
    allowed_uses.1:        "key encipherment" => "key encipherment"
    allowed_uses.2:        "server auth" => "server auth"
    ca_cert_pem:           "f6596b596ae22ec47779834d3235f7253049a64b" => "f6596b596ae22ec47779834d3235f7253049a64b"
    ca_key_algorithm:      "ECDSA" => "ECDSA"
    ca_private_key_pem:    "928191fb4925850f217b4858c2a2dbac3095de73" => "928191fb4925850f217b4858c2a2dbac3095de73"
    cert_pem:              "-----BEGIN CERTIFICATE-----\n ... \n-----END CERTIFICATE-----\n" => "<computed>"
    cert_request_pem:      "0b4c65dad766b494081c08f8d201d0a362dd2f8d" => "b99749122fd1790e86ea0a413a5a8d247fa231c9" (forces new resource)
    early_renewal_hours:   "0" => "0"
    validity_end_time:     "2026-09-08T19:01:06.751317848+08:00" => "<computed>"
    validity_period_hours: "87600" => "87600"
    validity_start_time:   "2016-09-10T19:01:06.751317848+08:00" => "<computed>"

The text was updated successfully, but these errors were encountered:

RichardKnop · 2016-09-10T11:32:35Z

Here is a sample Terraform file to replicate the problem.

terraform apply will always force new resource.

The only way to work around it is to pass -refresh=false flag. But that is not really a solution.

resource "tls_private_key" "ca" {
  algorithm = "ECDSA"
  ecdsa_curve = "P521"
}

resource "tls_self_signed_cert" "ca" {
  key_algorithm = "ECDSA"
  private_key_pem = "${tls_private_key.ca.private_key_pem}"

  subject {
    common_name = "My company CA"
    organization = "My company"
    country = "UK"
  }

  validity_period_hours = 87600 # 10 years

  is_ca_certificate = true

  allowed_uses = [
    "key_encipherment",
    "digital_signature",
    "server_auth",
    "client_auth",
    "cert_signing",
  ]
}

resource "tls_private_key" "server" {
  count = "3"
  algorithm = "ECDSA"
  ecdsa_curve = "P521"
}

resource "tls_private_key" "client_server" {
  count = "3"
  algorithm = "ECDSA"
  ecdsa_curve = "P521"
}

resource "tls_private_key" "client" {
  count = "3"
  algorithm = "ECDSA"
  ecdsa_curve = "P521"
}

data "tls_cert_request" "server" {
  count = "3"
  key_algorithm = "${element(tls_private_key.server.*.algorithm, count.index)}"
  private_key_pem = "${element(tls_private_key.server.*.private_key_pem, count.index)}"

  subject {
    common_name = "coreos${count.index}"
    organization = "My company"
    country = "UK"
  }

  dns_names = [
    # hostname
    "coreos${count.index}",
  ]
}

resource "tls_locally_signed_cert" "server" {
  count = "3"
  cert_request_pem = "${element(data.tls_cert_request.server.*.cert_request_pem, count.index)}"

  ca_key_algorithm = "ECDSA"
  ca_private_key_pem = "${tls_private_key.ca.private_key_pem}"
  ca_cert_pem = "${tls_self_signed_cert.ca.cert_pem}"

  validity_period_hours = 87600 # 10 years

  allowed_uses = [
    "signing",
    "key encipherment",
    "server auth",
  ]
}

data "tls_cert_request" "client_server" {
  count = "3"
  key_algorithm = "${element(tls_private_key.client_server.*.algorithm, count.index)}"
  private_key_pem = "${element(tls_private_key.client_server.*.private_key_pem, count.index)}"

  subject {
    common_name = "coreos${count.index}"
    organization = "My company"
    country = "UK"
  }

  dns_names = [
    # hostname
    "coreos${count.index}",
  ]
}

resource "tls_locally_signed_cert" "client_server" {
  count = "3"
  cert_request_pem = "${element(data.tls_cert_request.client_server.*.cert_request_pem, count.index)}"

  ca_key_algorithm = "ECDSA"
  ca_private_key_pem = "${tls_private_key.ca.private_key_pem}"
  ca_cert_pem = "${tls_self_signed_cert.ca.cert_pem}"

  validity_period_hours = 87600 # 10 years

  allowed_uses = [
    "signing",
    "key encipherment",
    "server auth",
    "client auth",
  ]
}

data "tls_cert_request" "client" {
  count = "3"
  key_algorithm = "${element(tls_private_key.client.*.algorithm, count.index)}"
  private_key_pem = "${element(tls_private_key.client.*.private_key_pem, count.index)}"

  subject {
    common_name = "coreos${count.index}"
    organization = "My company"
    country = "UK"
  }

  dns_names = [
    # hostname
    "coreos${count.index}",
  ]
}

resource "tls_locally_signed_cert" "client" {
  count = "3"
  cert_request_pem = "${element(data.tls_cert_request.client.*.cert_request_pem, count.index)}"

  ca_key_algorithm = "ECDSA"
  ca_private_key_pem = "${tls_private_key.ca.private_key_pem}"
  ca_cert_pem = "${tls_self_signed_cert.ca.cert_pem}"

  validity_period_hours = 87600 # 10 years

  allowed_uses = [
    "signing",
    "key encipherment",
    "client auth",
  ]
}

apparentlymart · 2016-09-10T20:09:10Z

I made a quick local build of Terraform that doesn't hash the cert_request_pem attribute for storage in state, so I could see more easily what was changing in the diffs. Here's one example:

    cert_request_pem:      "-----BEGIN CERTIFICATE REQUEST-----\nMIIByTCCASoCAQAwYDELMAkGA1UEBhMCVUsxCTAHBgNVBAgTADEJMAcGA1UEBxMA\nMQkwBwYDVQQREwAxEzARBgNVBAoTCk15IGNvbXBhbnkxCTAHBgNVBAsTADEQMA4G\nA1UEAxMHY29yZW9zMjCBmzAQBgcqhkjOPQIBBgUrgQQAIwOBhgAEAQNrPsnCWzxp\nBVBfKtICnfRkml21oTdeITbx2FnjF5tVBNVUUj7jAp4c7jVF2MyU9RvIIulFKuVs\nVT7eSjrehhEkAY2vsXfSOoHSvlFpxjezTc+RgQbqRB1u3DYYEXXf3DhDWIKhLvEM\n9z6iMh4gM5+m/2UIN8WcZP2YKD2JrKwsQJO4oCUwIwYJKoZIhvcNAQkOMRYwFDAS\nBgNVHREECzAJggdjb3Jlb3MyMAoGCCqGSM49BAMEA4GMADCBiAJCAdeWzIWHsUFj\nrZwpKJZ0poWp8Uzl19oSlmpWCJl8iqt6fZw8yYUlhhgVNFWxMThxqW+dtutloZMl\nbV1iivNPTxn3AkIBldFiy7lRcq05oo1124w8f6OED2CA2ZjGx/oaTemyO2oa/9/E\n+YDUclt9oY389HjxclCyElApSwkvoff68nFmKiI=\n-----END CERTIFICATE REQUEST-----" => "-----BEGIN CERTIFICATE REQUEST-----\nMIIByDCCASoCAQAwYDELMAkGA1UEBhMCVUsxCTAHBgNVBAgTADEJMAcGA1UEBxMA\nMQkwBwYDVQQREwAxEzARBgNVBAoTCk15IGNvbXBhbnkxCTAHBgNVBAsTADEQMA4G\nA1UEAxMHY29yZW9zMjCBmzAQBgcqhkjOPQIBBgUrgQQAIwOBhgAEAQNrPsnCWzxp\nBVBfKtICnfRkml21oTdeITbx2FnjF5tVBNVUUj7jAp4c7jVF2MyU9RvIIulFKuVs\nVT7eSjrehhEkAY2vsXfSOoHSvlFpxjezTc+RgQbqRB1u3DYYEXXf3DhDWIKhLvEM\n9z6iMh4gM5+m/2UIN8WcZP2YKD2JrKwsQJO4oCUwIwYJKoZIhvcNAQkOMRYwFDAS\nBgNVHREECzAJggdjb3Jlb3MyMAoGCCqGSM49BAMEA4GLADCBhwJCARDpIG22E9r+\nT1Prt07FjnEOVTFEoYdoZfSXQbZjPC2cmz2ydrax/UVJR7Im6lRQqD2oeC6FmaCQ\nqIgVCrTyupgRAkEPes/Ovnw114PDJwN1vgAVQxC3D2gwMr6egOgZxnc6ZUt3AuFY\nb1WVK8J/8u1jjIblT2JeaPs5LpGDF72qABoJkA==\n-----END CERTIFICATE REQUEST-----" (forces new resource)

Extracting these in a readable way, we have the following two cert requests:

These two are different, so the diff is warranted. Of course, the PEM encoding makes it hard to see what exactly is going on here, so we can decode it:

Certificate Request:
    Data:
        Version: 0 (0x0)
        Subject: C=UK, ST=, L=/postalCode=, O=My company, OU=, CN=coreos2
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (521 bit)
                pub: 
                    04:01:03:6b:3e:c9:c2:5b:3c:69:05:50:5f:2a:d2:
                    02:9d:f4:64:9a:5d:b5:a1:37:5e:21:36:f1:d8:59:
                    e3:17:9b:55:04:d5:54:52:3e:e3:02:9e:1c:ee:35:
                    45:d8:cc:94:f5:1b:c8:22:e9:45:2a:e5:6c:55:3e:
                    de:4a:3a:de:86:11:24:01:8d:af:b1:77:d2:3a:81:
                    d2:be:51:69:c6:37:b3:4d:cf:91:81:06:ea:44:1d:
                    6e:dc:36:18:11:75:df:dc:38:43:58:82:a1:2e:f1:
                    0c:f7:3e:a2:32:1e:20:33:9f:a6:ff:65:08:37:c5:
                    9c:64:fd:98:28:3d:89:ac:ac:2c:40:93:b8
                ASN1 OID: secp521r1
                NIST CURVE: P-521
        Attributes:
        Requested Extensions:
            X509v3 Subject Alternative Name: 
                DNS:coreos2
    Signature Algorithm: ecdsa-with-SHA512
         30:81:88:02:42:01:d7:96:cc:85:87:b1:41:63:ad:9c:29:28:
         96:74:a6:85:a9:f1:4c:e5:d7:da:12:96:6a:56:08:99:7c:8a:
         ab:7a:7d:9c:3c:c9:85:25:86:18:15:34:55:b1:31:38:71:a9:
         6f:9d:b6:eb:65:a1:93:25:6d:5d:62:8a:f3:4f:4f:19:f7:02:
         42:01:95:d1:62:cb:b9:51:72:ad:39:a2:8d:75:db:8c:3c:7f:
         a3:84:0f:60:80:d9:98:c6:c7:fa:1a:4d:e9:b2:3b:6a:1a:ff:
         df:c4:f9:80:d4:72:5b:7d:a1:8d:fc:f4:78:f1:72:50:b2:12:
         50:29:4b:09:2f:a1:f7:fa:f2:71:66:2a:22

Certificate Request:
    Data:
        Version: 0 (0x0)
        Subject: C=UK, ST=, L=/postalCode=, O=My company, OU=, CN=coreos2
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (521 bit)
                pub: 
                    04:01:03:6b:3e:c9:c2:5b:3c:69:05:50:5f:2a:d2:
                    02:9d:f4:64:9a:5d:b5:a1:37:5e:21:36:f1:d8:59:
                    e3:17:9b:55:04:d5:54:52:3e:e3:02:9e:1c:ee:35:
                    45:d8:cc:94:f5:1b:c8:22:e9:45:2a:e5:6c:55:3e:
                    de:4a:3a:de:86:11:24:01:8d:af:b1:77:d2:3a:81:
                    d2:be:51:69:c6:37:b3:4d:cf:91:81:06:ea:44:1d:
                    6e:dc:36:18:11:75:df:dc:38:43:58:82:a1:2e:f1:
                    0c:f7:3e:a2:32:1e:20:33:9f:a6:ff:65:08:37:c5:
                    9c:64:fd:98:28:3d:89:ac:ac:2c:40:93:b8
                ASN1 OID: secp521r1
                NIST CURVE: P-521
        Attributes:
        Requested Extensions:
            X509v3 Subject Alternative Name: 
                DNS:coreos2
    Signature Algorithm: ecdsa-with-SHA512
         30:81:87:02:42:01:10:e9:20:6d:b6:13:da:fe:4f:53:eb:b7:
         4e:c5:8e:71:0e:55:31:44:a1:87:68:65:f4:97:41:b6:63:3c:
         2d:9c:9b:3d:b2:76:b6:b1:fd:45:49:47:b2:26:ea:54:50:a8:
         3d:a8:78:2e:85:99:a0:90:a8:88:15:0a:b4:f2:ba:98:11:02:
         41:0f:7a:cf:ce:be:7c:35:d7:83:c3:27:03:75:be:00:15:43:
         10:b7:0f:68:30:32:be:9e:80:e8:19:c6:77:3a:65:4b:77:02:
         e1:58:6f:55:95:2b:c2:7f:f2:ed:63:8c:86:e5:4f:62:5e:68:
         fb:39:2e:91:83:17:bd:aa:00:1a:09:90

Here's the diff:

@@ -22,11 +22,11 @@
             X509v3 Subject Alternative Name: 
                 DNS:coreos2
     Signature Algorithm: ecdsa-with-SHA512
-         30:81:88:02:42:01:d7:96:cc:85:87:b1:41:63:ad:9c:29:28:
-         96:74:a6:85:a9:f1:4c:e5:d7:da:12:96:6a:56:08:99:7c:8a:
-         ab:7a:7d:9c:3c:c9:85:25:86:18:15:34:55:b1:31:38:71:a9:
-         6f:9d:b6:eb:65:a1:93:25:6d:5d:62:8a:f3:4f:4f:19:f7:02:
-         42:01:95:d1:62:cb:b9:51:72:ad:39:a2:8d:75:db:8c:3c:7f:
-         a3:84:0f:60:80:d9:98:c6:c7:fa:1a:4d:e9:b2:3b:6a:1a:ff:
-         df:c4:f9:80:d4:72:5b:7d:a1:8d:fc:f4:78:f1:72:50:b2:12:
-         50:29:4b:09:2f:a1:f7:fa:f2:71:66:2a:22
+         30:81:87:02:42:01:10:e9:20:6d:b6:13:da:fe:4f:53:eb:b7:
+         4e:c5:8e:71:0e:55:31:44:a1:87:68:65:f4:97:41:b6:63:3c:
+         2d:9c:9b:3d:b2:76:b6:b1:fd:45:49:47:b2:26:ea:54:50:a8:
+         3d:a8:78:2e:85:99:a0:90:a8:88:15:0a:b4:f2:ba:98:11:02:
+         41:0f:7a:cf:ce:be:7c:35:d7:83:c3:27:03:75:be:00:15:43:
+         10:b7:0f:68:30:32:be:9e:80:e8:19:c6:77:3a:65:4b:77:02:
+         e1:58:6f:55:95:2b:c2:7f:f2:ed:63:8c:86:e5:4f:62:5e:68:
+         fb:39:2e:91:83:17:bd:aa:00:1a:09:90

So the part that changed here was the signature on the CSR. This makes sense, since a signature contains a randomly-generated nonce.

Seems like I was a bit over-eager to data-source-ize this resource, since I failed to consider that it had a random element embedded in it. In Terraform our pattern is to treat random numbers as a resource so that we can hold on to them in the state after they have been generated.

I think the fix here would be to turn tls_cert_request back into a managed resource. For the moment my suggested workaround would be to use a resource block to declare these, which should work in spite of the deprecation warnings that Terraform will generate. Then we can fix this by removing that deprecation warning and making the data source be the deprecated element, since a data source that doesn't generate a stable result isn't useful.

I'll accept full responsibility for this oversight; in my vigor to convert all of the "read only" things to data sources for the 0.7 release I didn't properly consider the implications of converting this one.

RichardKnop · 2016-09-11T11:15:53Z

@apparentlymart Thanks for looking into this. Your proposed solution makes sense 👍

I will revert back to using resources for now and ignore the deprecation warnings and then once tls_cert_request is reverted back from data source to resource the warnings should disappear.

RichardKnop · 2016-09-11T13:32:03Z

@apparentlymart Btw, this is still an issue. It shouldn't be closed as it still happens in the latest dev version.

#7821

ghost · 2020-04-21T02:37:22Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

apparentlymart added the provider/tls label Sep 10, 2016

apparentlymart self-assigned this Sep 10, 2016

This was referenced Sep 12, 2016

tls_cert_request causes refresh to fail #8782

Closed

tls_cert_request forces recreating resources when count changes #8781

Closed

apparentlymart mentioned this issue Sep 24, 2016

Restore tls_cert_request to being a managed resource #9035

Merged

jen20 closed this as completed in #9035 Sep 30, 2016

ghost locked and limited conversation to collaborators Apr 21, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tls_cert_request with count forces new resource every time #8764

tls_cert_request with count forces new resource every time #8764

RichardKnop commented Sep 10, 2016

RichardKnop commented Sep 10, 2016 •

edited

Loading

apparentlymart commented Sep 10, 2016

RichardKnop commented Sep 11, 2016

RichardKnop commented Sep 11, 2016

ghost commented Apr 21, 2020

tls_cert_request with count forces new resource every time #8764

tls_cert_request with count forces new resource every time #8764

Comments

RichardKnop commented Sep 10, 2016

Terraform Version

Affected Resource(s)

Debug Output

RichardKnop commented Sep 10, 2016 • edited Loading

apparentlymart commented Sep 10, 2016

RichardKnop commented Sep 11, 2016

RichardKnop commented Sep 11, 2016

ghost commented Apr 21, 2020

RichardKnop commented Sep 10, 2016 •

edited

Loading