Add warnings about custom role format for IAM bindings

.github/ISSUE_TEMPLATE/bug.md

@@ -9,10 +9,10 @@ about: For when something is there, but doesn't work how it should.
 
 ### Community Note
 
-* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
-* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
-* If you are interested in working on this issue or have submitted a pull request, please leave a comment
-* If an issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to "hashibot", a community member has claimed the issue already.
+* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request.
+* Please do not leave _+1_ or _me too_ comments, they generate extra noise for issue followers and do not help prioritize the request.
+* If you are interested in working on this issue or have submitted a pull request, please leave a comment.
+* If an issue is assigned to the `modular-magician` user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to `hashibot`, a community member has claimed the issue already.
 
 <!--- Thank you for keeping this note for the community --->
 
@@ -31,12 +31,16 @@ about: For when something is there, but doesn't work how it should.
 <!--- Information about code formatting: https://help.github.com/articles/basic-writing-and-formatting-syntax/#quoting-code --->
 
 ```tf
-# Copy-paste your Terraform configurations here - for large Terraform configs,
-# please use a service like Dropbox and share a link to the ZIP file. For
-# security, you can also encrypt the files using our GPG public key: https://www.hashicorp.com/security
+# Copy-paste your Terraform configurations here.
+#
+# For large Terraform configs, please use a service like Dropbox and share a link to the ZIP file.
+# For security, you can also encrypt the files using our GPG public key:
+#    https://www.hashicorp.com/security
+#
 # If reproducing the bug involves modifying the config file (e.g., apply a config,
-# change a value, apply the config again, see the bug) then please include both the
-# version of the config before the change, and the version of the config after the change.
+# change a value, apply the config again, see the bug), then please include both:
+# * the version of the config before the change, and
+# * the version of the config after the change.
 ```
 
 ### Debug Output

CHANGELOG.md

@@ -1,4 +1,48 @@
-## 3.3.1 (Unreleased)
+## 3.4.1 (Unreleased)
+## 3.4.0 (January 07, 2020)
+
+DEPRECATIONS:
+* kms: deprecated `data.google_kms_secret_ciphertext` as there was no way to make it idempotent. Instead, use the `google_kms_secret_ciphertext` resource. ([#5314](https://github.com/terraform-providers/terraform-provider-google/pull/5314))
+
+BREAKING CHANGES:
+* cloudrun: Changed `google_cloud_run_domain_mapping` to correctly match Cloud Run API expected format for `spec.route_name`, {serviceName}, instead of invalid projects/{project}/global/services/{serviceName} ([#5264](https://github.com/terraform-providers/terraform-provider-google/pull/5264))
+* compute: Added back ConflictsWith restrictions for ExactlyOneOf restrictions that were removed in v3.3.0 for `google_compute_firewall`, `google_compute_health_check`, and `google_compute_region_health_check`. This effectively changes an API-side failure that was only accessible in v3.3.0 to a plan-time one. ([#5220](https://github.com/terraform-providers/terraform-provider-google/pull/5220))
+* logging: Changed `google_logging_metric.metric_descriptors.labels` from a list to a set ([#5258](https://github.com/terraform-providers/terraform-provider-google/pull/5258))
+* resourcemanager: Added back ConflictsWith restrictions for ExactlyOneOf restrictions that were removed in v3.3.0 for `google_organization_policy`, `google_folder_organization_policy`, and `google_project_organization_policy`. This effectively changes an API-side failure that was only accessible in v3.3.0 to a plan-time one. ([#5220](https://github.com/terraform-providers/terraform-provider-google/pull/5220))
+
+FEATURES:
+* **New Data Source:** google_sql_ca_certs ([#5306](https://github.com/terraform-providers/terraform-provider-google/pull/5306))
+* **New Resource:** `google_identity_platform_default_supported_idp_config` ([#5199](https://github.com/terraform-providers/terraform-provider-google/pull/5199))
+* **New Resource:** `google_identity_platform_inbound_saml_config` ([#5199](https://github.com/terraform-providers/terraform-provider-google/pull/5199))
+* **New Resource:** `google_identity_platform_oauth_idp_config` ([#5199](https://github.com/terraform-providers/terraform-provider-google/pull/5199))
+* **New Resource:** `google_identity_platform_tenant_default_supported_idp_config` ([#5199](https://github.com/terraform-providers/terraform-provider-google/pull/5199))
+* **New Resource:** `google_identity_platform_tenant_inbound_saml_config` ([#5199](https://github.com/terraform-providers/terraform-provider-google/pull/5199))
+* **New Resource:** `google_identity_platform_tenant_oauth_idp_config` ([#5199](https://github.com/terraform-providers/terraform-provider-google/pull/5199))
+* **New Resource:** `google_identity_platform_tenant` ([#5199](https://github.com/terraform-providers/terraform-provider-google/pull/5199))
+* **New Resource:** `google_kms_crypto_key_iam_policy` ([#5247](https://github.com/terraform-providers/terraform-provider-google/pull/5247))
+* **New Resource:** `google_kms_secret_ciphertext` ([#5314](https://github.com/terraform-providers/terraform-provider-google/pull/5314))
+
+IMPROVEMENTS:
+* composer: Increased default timeouts for `google_composer_environment` ([#5223](https://github.com/terraform-providers/terraform-provider-google/pull/5223))
+* compute: Added graceful termination to `container_cluster` create calls so that partially created clusters will resume the original operation if the Terraform process is killed mid create. ([#5217](https://github.com/terraform-providers/terraform-provider-google/pull/5217))
+* compute: Fixed `google_compute_disk_resource_policy_attachment` parsing of region from zone to allow for provider-level zone and make error message more accurate` ([#5257](https://github.com/terraform-providers/terraform-provider-google/pull/5257))
+* provider: Reduced default `send_after` controlling the time interval after which a batched request sends. ([#5268](https://github.com/terraform-providers/terraform-provider-google/pull/5268))
+
+BUG FIXES:
+* all: fixed issue where many fields that were removed in 3.0.0 would show a diff when they were removed from config ([#5313](https://github.com/terraform-providers/terraform-provider-google/pull/5313))
+* bigquery: fixed `bigquery_table.encryption_configuration` to correctly recreate the table when modified ([#5321](https://github.com/terraform-providers/terraform-provider-google/pull/5321))
+* cloudrun:  Changed `google_cloud_run_domain_mapping` to correctly match Cloud Run API expected format for `spec.route_name`, {serviceName}, instead of invalid projects/{project}/global/services/{serviceName} ([#5264](https://github.com/terraform-providers/terraform-provider-google/pull/5264))
+* cloudrun: Changed `cloud_run_domain_mapping` to poll for success or failure and throw an appropriate error when ready status returns as false. ([#5267](https://github.com/terraform-providers/terraform-provider-google/pull/5267))
+* cloudrun: Fixed `google_cloudrun_service` to allow update instead of force-recreation for changes in `spec` `env` and `command` fields ([#5269](https://github.com/terraform-providers/terraform-provider-google/pull/5269))
+* cloudrun: Removed unsupported update for `google_cloud_run_domain_mapping` to allow force-recreation. ([#5253](https://github.com/terraform-providers/terraform-provider-google/pull/5253))
+* cloudrun: Stopped returning an error when a `cloud_run_domain_mapping` was waiting on DNS verification. ([#5315](https://github.com/terraform-providers/terraform-provider-google/pull/5315))
+* compute: Fixed `google_compute_backend_service` to allow updating `cdn_policy.cache_key_policy.*` fields to false or empty. ([#5276](https://github.com/terraform-providers/terraform-provider-google/pull/5276))
+* compute: Fixed behaviour where `google_compute_subnetwork` did not record a value for `name` when `self_link` was specified. ([#5288](https://github.com/terraform-providers/terraform-provider-google/pull/5288))
+* container: fixed issue where an empty variable in `tags` would cause a crash ([#5226](https://github.com/terraform-providers/terraform-provider-google/pull/5226))
+* endpoints: Added operation wait for `google_endpoints_service` to fix 403 "Service not found" errors during initial creation ([#5259](https://github.com/terraform-providers/terraform-provider-google/pull/5259))
+* logging: Made `google_logging_metric.metric_descriptors.labels` a set to prevent diff from ordering ([#5258](https://github.com/terraform-providers/terraform-provider-google/pull/5258))
+* resourcemanager: added retries for `data.google_organization` ([#5246](https://github.com/terraform-providers/terraform-provider-google/pull/5246))
+
 ## 3.3.0 (December 17, 2019)
 
 FEATURES:

google/bootstrap_utils_test.go

@@ -6,6 +6,7 @@ import (
 	"log"
 	"os"
 	"testing"
+	"time"
 
 	"google.golang.org/api/cloudkms/v1"
 	"google.golang.org/api/iam/v1"
@@ -230,3 +231,63 @@ func BootstrapServiceAccount(t *testing.T, project, testRunner string) string {
 
 	return sa.Email
 }
+
+const SharedTestNetworkPrefix = "tf-bootstrap-net-"
+
+// BootstrapSharedServiceNetworkingConsumerNetwork will return a shared compute network
+// for service networking test to prevent hitting limits on tenancy projects.
+//
+// This will either return an existing network or create one if it hasn't been created
+// in the project yet. One consumer network/tenant project we don't own is created
+// per producer network (i.e. network created by test), with a hard limit set.
+func BootstrapSharedServiceNetworkingConsumerNetwork(t *testing.T, testId string) string {
+	if v := os.Getenv("TF_ACC"); v == "" {
+		log.Println("Acceptance tests and bootstrapping skipped unless env 'TF_ACC' set")
+		// If not running acceptance tests, return an empty string
+		return ""
+	}
+
+	project := getTestProjectFromEnv()
+	networkName := SharedTestNetworkPrefix + testId
+	config := &Config{
+		Credentials: getTestCredsFromEnv(),
+		Project:     project,
+		Region:      getTestRegionFromEnv(),
+		Zone:        getTestZoneFromEnv(),
+	}
+	ConfigureBasePaths(config)
+	if err := config.LoadAndValidate(context.Background()); err != nil {
+		t.Errorf("Unable to bootstrap network: %s", err)
+	}
+
+	log.Printf("[DEBUG] Getting shared test network %q", networkName)
+	_, err := config.clientCompute.Networks.Get(project, networkName).Do()
+	if err != nil && isGoogleApiErrorWithCode(err, 404) {
+		log.Printf("[DEBUG] Network %q not found, bootstrapping", networkName)
+		url := fmt.Sprintf("%sprojects/%s/global/networks", config.ComputeBasePath, project)
+		netObj := map[string]interface{}{
+			"name":                  networkName,
+			"autoCreateSubnetworks": false,
+		}
+
+		res, err := sendRequestWithTimeout(config, "POST", project, url, netObj, 4*time.Minute)
+		if err != nil {
+			t.Fatalf("Error bootstrapping shared test network %q: %s", networkName, err)
+		}
+
+		log.Printf("[DEBUG] Waiting for network creation to finish")
+		err = computeOperationWaitTime(config, res, project, "Error bootstrapping shared test network", 4)
+		if err != nil {
+			t.Fatalf("Error bootstrapping shared test network %q: %s", networkName, err)
+		}
+	}
+
+	network, err := config.clientCompute.Networks.Get(project, networkName).Do()
+	if err != nil {
+		t.Errorf("Error getting shared test network %q: %s", networkName, err)
+	}
+	if network == nil {
+		t.Fatalf("Error getting shared test network %q: is nil", networkName)
+	}
+	return network.Name
+}

google/cloudrun_polling.go

@@ -59,6 +59,11 @@ func (s KnativeStatus) LatestMessage() string {
 func (s KnativeStatus) State(res interface{}) string {
 	for _, condition := range s.Status.Conditions {
 		if condition.Type == "Ready" {
+			// DomainMapping can enter a 'terminal' state of waiting for external verification
+			// of DNS records.
+			if condition.Reason == "CertificatePending" {
+				return "Ready:CertificatePending"
+			}
 			return fmt.Sprintf("%s:%s", condition.Type, condition.Status)
 		}
 	}
@@ -76,7 +81,7 @@ func (p *CloudRunPolling) PendingStates() []string {
 	return []string{"Ready:Unknown", "Empty"}
 }
 func (p *CloudRunPolling) TargetStates() []string {
-	return []string{"Ready:True"}
+	return []string{"Ready:True", "Ready:CertificatePending"}
 }
 func (p *CloudRunPolling) ErrorStates() []string {
 	return []string{"Ready:False"}

google/data_source_google_kms_secret_ciphertext.go

@@ -13,7 +13,8 @@ import (
 
 func dataSourceGoogleKmsSecretCiphertext() *schema.Resource {
 	return &schema.Resource{
-		Read: dataSourceGoogleKmsSecretCiphertextRead,
+		DeprecationMessage: "Use the google_kms_secret_ciphertext resource instead.",
+		Read:               dataSourceGoogleKmsSecretCiphertextRead,
 		Schema: map[string]*schema.Schema{
 			"crypto_key": {
 				Type:     schema.TypeString,

google/data_source_google_kms_secret_ciphertext_test.go

@@ -1,113 +1,41 @@
 package google
 
 import (
-	"encoding/base64"
 	"fmt"
-	"log"
 	"testing"
 
 	"github.com/hashicorp/terraform-plugin-sdk/helper/acctest"
 	"github.com/hashicorp/terraform-plugin-sdk/helper/resource"
 	"github.com/hashicorp/terraform-plugin-sdk/terraform"
-	"google.golang.org/api/cloudkms/v1"
 )
 
-func TestAccKmsSecretCiphertext_basic(t *testing.T) {
+func TestAccDataKmsSecretCiphertext_basic(t *testing.T) {
 	t.Parallel()
 
-	projectOrg := getTestOrgFromEnv(t)
-	projectBillingAccount := getTestBillingAccountFromEnv(t)
-
-	projectId := "terraform-" + acctest.RandString(10)
-	keyRingName := fmt.Sprintf("tf-test-%s", acctest.RandString(10))
-	cryptoKeyName := fmt.Sprintf("tf-test-%s", acctest.RandString(10))
+	kms := BootstrapKMSKey(t)
 
 	plaintext := fmt.Sprintf("secret-%s", acctest.RandString(10))
 
-	// The first test creates resources needed to encrypt plaintext and produce ciphertext
 	resource.Test(t, resource.TestCase{
 		PreCheck:  func() { testAccPreCheck(t) },
 		Providers: testAccProviders,
 		Steps: []resource.TestStep{
 			{
-				Config: testGoogleKmsCryptoKey_basic(projectId, projectOrg, projectBillingAccount, keyRingName, cryptoKeyName),
+				Config: testGoogleKmsSecretCiphertext_datasource(kms.CryptoKey.Name, plaintext),
 				Check: func(s *terraform.State) error {
-					cryptoKeyId, err := getCryptoKeyId(s, "google_kms_crypto_key.crypto_key")
+					plaintext, err := testAccDecryptSecretDataWithCryptoKey(s, kms.CryptoKey.Name, "data.google_kms_secret_ciphertext.acceptance")
 
 					if err != nil {
 						return err
 					}
 
-					// The second test asserts that the data source created a ciphertext that can be decrypted to the correct plaintext
-					resource.Test(t, resource.TestCase{
-						PreCheck:  func() { testAccPreCheck(t) },
-						Providers: testAccProviders,
-						Steps: []resource.TestStep{
-							{
-								Config: testGoogleKmsSecretCiphertext_datasource(cryptoKeyId.terraformId(), plaintext),
-								Check: func(s *terraform.State) error {
-									plaintext, err := testAccDecryptSecretDataWithCryptoKey(s, cryptoKeyId, "data.google_kms_secret_ciphertext.acceptance")
-
-									if err != nil {
-										return err
-									}
-
-									return resource.TestCheckResourceAttr("data.google_kms_secret_ciphertext.acceptance", "plaintext", plaintext)(s)
-								},
-							},
-						},
-					})
-
-					return nil
+					return resource.TestCheckResourceAttr("data.google_kms_secret_ciphertext.acceptance", "plaintext", plaintext)(s)
 				},
 			},
 		},
 	})
 }
 
-func getCryptoKeyId(s *terraform.State, cryptoKeyResourceName string) (*kmsCryptoKeyId, error) {
-	config := testAccProvider.Meta().(*Config)
-	rs, ok := s.RootModule().Resources[cryptoKeyResourceName]
-	if !ok {
-		return nil, fmt.Errorf("Resource not found: %s", cryptoKeyResourceName)
-	}
-
-	return parseKmsCryptoKeyId(rs.Primary.Attributes["id"], config)
-}
-
-func testAccDecryptSecretDataWithCryptoKey(s *terraform.State, cryptoKeyId *kmsCryptoKeyId, secretCiphertextResourceName string) (string, error) {
-	config := testAccProvider.Meta().(*Config)
-	rs, ok := s.RootModule().Resources[secretCiphertextResourceName]
-	if !ok {
-		return "", fmt.Errorf("Resource not found: %s", secretCiphertextResourceName)
-	}
-	ciphertext, ok := rs.Primary.Attributes["ciphertext"]
-	if !ok {
-		return "", fmt.Errorf("Attribute 'ciphertext' not found in resource '%s'", secretCiphertextResourceName)
-	}
-
-	kmsDecryptRequest := &cloudkms.DecryptRequest{
-		Ciphertext: ciphertext,
-	}
-
-	decryptResponse, err := config.clientKms.Projects.Locations.KeyRings.CryptoKeys.Decrypt(cryptoKeyId.cryptoKeyId(), kmsDecryptRequest).Do()
-
-	if err != nil {
-		return "", fmt.Errorf("Error decrypting ciphertext: %s", err)
-	}
-
-	plaintextBytes, err := base64.StdEncoding.DecodeString(decryptResponse.Plaintext)
-
-	if err != nil {
-		return "", err
-	}
-
-	plaintext := string(plaintextBytes)
-	log.Printf("[INFO] Successfully decrypted ciphertext and got plaintext: %s", plaintext)
-
-	return plaintext, nil
-}
-
 func testGoogleKmsSecretCiphertext_datasource(cryptoKeyTerraformId, plaintext string) string {
 	return fmt.Sprintf(`
 data "google_kms_secret_ciphertext" "acceptance" {

google/provider.go

@@ -477,9 +477,9 @@ func Provider() terraform.ResourceProvider {
 	return provider
 }
 
-// Generated resources: 95
+// Generated resources: 96
 // Generated IAM resources: 45
-// Total generated resources: 140
+// Total generated resources: 141
 func ResourceMap() map[string]*schema.Resource {
 	resourceMap, _ := ResourceMapWithErrors()
 	return resourceMap
@@ -597,6 +597,7 @@ func ResourceMapWithErrors() (map[string]*schema.Resource, error) {
 			"google_identity_platform_tenant":                              resourceIdentityPlatformTenant(),
 			"google_kms_key_ring":                                          resourceKMSKeyRing(),
 			"google_kms_crypto_key":                                        resourceKMSCryptoKey(),
+			"google_kms_secret_ciphertext":                                 resourceKMSSecretCiphertext(),
 			"google_logging_metric":                                        resourceLoggingMetric(),
 			"google_ml_engine_model":                                       resourceMLEngineModel(),
 			"google_monitoring_alert_policy":                               resourceMonitoringAlertPolicy(),

google/resource_bigtable_gc_policy.go

@@ -151,7 +151,7 @@ func resourceBigtableGCPolicyRead(d *schema.ResourceData, meta interface{}) erro
 	if err != nil {
 		log.Printf("[WARN] Removing %s because it's gone", name)
 		d.SetId("")
-		return fmt.Errorf("Error retrieving table. Could not find %s in %s. %s", name, instanceName, err)
+		return nil
 	}
 
 	for _, fi := range ti.FamilyInfos {