ci: Initial investigation of Terraform in Kokoro integration tests

.cloud/.gitignore

@@ -0,0 +1,7 @@
+generated.tfplan
+generated.tfplan.json
+generated.auto.tfvars
+generated-env.sh
+generated-main.tf
+generated-outputs.tf
+generated-variables.tf

.cloud/README.md

@@ -0,0 +1,149 @@
+# Introduction
+
+When you experience integration test failures, they may be due to concurrent tests accessing the
+same GCP project resources, required resources not existing on the GCP project, quota limits being
+exceeded due to overreliance on a single project, incorrect permissions, disabled APIs, and/or
+service account impersonation requirements.
+
+[Terraform](https://www.terraform.io/) is an "Infrastructure as Code" (IaC) tool that allows cloud
+and on-prem resources to be defined in configuration files to be versioned, reused, and shared. By
+using Terraform to provision our GCP projects prior to running integration tests we:
+
+* Ensure infrastructure provisioning consistency,
+* Prevent conflicts between concurrent integration test invocations using the same GCP project by
+  randomizing resource names,
+* Provide a managed lifecycle including the ability to destroy short-lived testing environments,
+* Version the infrastructure requirements with the test source.
+
+# Running `google-cloud-java` Integration Tests
+
+When you need to troubleshoot integration tests in the google-cloud-java repository, follow one of
+the supported workflows below to provision a GCP project with the required configuration.
+
+## Installation
+
+1. Download and install the latest version of Terraform: https://www.terraform.io/downloads
+2. *(Optional)* Update your default shell environment variables. (For Mac users,
+   modify `~/.bash_profile`.
+   For Linux users, modify `~/.bashrc`.)
+    * See
+      also: [About Cloud Billing accounts](https://cloud.google.com/billing/docs/how-to/manage-billing-account)
+    * See
+      also: [Creating and managing Folders](https://cloud.google.com/resource-manager/docs/creating-managing-folders)
+
+```shell                       
+export GOOGLE_CLOUD_FOLDER_ID="000000000000"
+export GOOGLE_CLOUD_BILLING_ACCOUNT="000000-000000-000000"
+export GOOGLE_CLOUD_PROJECT_PREFIX="my-project"
+```
+
+## Supported Workflows
+
+During these workflows, you may be asked to authenticate one or two times with gcloud as it
+optionally creates a GCP project, and then sets up the
+[application-default credentials](https://cloud.google.com/docs/authentication/application-default-credentials)
+to be used by Terraform and the client libraries.
+
+A service account will be created, assigned `roles/owner`[^1], and used for all client library
+integration tests. Your user credentials must have permission in the designated GCP project to
+self-assign `roles/iam.serviceAccountTokenCreator` to impersonate the service account.
+
+[^1]: Basic roles like `roles/owner` are
+[not recommended in production environments](https://cloud.google.com/iam/docs/understanding-roles#basic)
+.
+
+### Workflow: Set up a GCP Project for **One or More** Client Libraries, then Test
+
+1. Invoke `setup.sh` with the directory names of the selected client libraries:
+   ```shell
+   $ ./.cloud/setup.sh [library][,library ...]
+   ```
+
+   Examples:
+    * `./.cloud/setup.sh java-accessapproval`
+    * `./.cloud/setup.sh java-asset,java-compute,java-container`
+
+2. When ready to begin testing, invoke `verify.sh`:
+   ```shell
+   $ ./.cloud/verify.sh [library][,library ...]
+   ```
+
+   Examples:
+    * `./.cloud/verify.sh java-accessapproval`
+    * `./.cloud/verify.sh java-asset,java-compute,java-container`
+
+Note: Invoking `verify.sh` without arguments will begin performing integration testing on all client
+libraries. In this workflow, the GCP project will only have been configured the project for a subset
+of client libraries, this command is unlikely to succeed unless the GCP project has been previously
+resourced and configured manually. If you wish to perform integration testing on all client
+libraries, use the below workflow to first provision the project for all client libraries.
+
+### Workflow: Set up a GCP Project for **All** Client Libraries, then Test
+
+1. Invoke `setup.sh` without arguments:
+   ```shell
+   $ ./.cloud/setup.sh
+   ```
+   Every client library with a `.cloud` subdirectory will be included in the generated Terraform
+   root module, and the GCP project will be prepared to test all client libraries.
+2. When ready to begin testing, invoke `verify.sh` with or without arguments:
+   ```shell
+   $ ./.cloud/verify.sh
+   ```
+   The above will perform integration tests on all client libraries.
+   ```shell
+   $ ./.cloud/verify.sh [library][,library ...]
+   ```
+   The above will only perform integration testing on the specified client libraries.
+
+   Examples:
+    * `./.cloud/verify.sh`
+    * `./.cloud/verify.sh java-accessapproval`
+    * `./.cloud/verify.sh java-asset,java-compute,java-container`
+
+### Workflow: Set up and Test with One Command
+
+Use `test.sh` to perform a combination of both `setup.sh` and `verify.sh`.
+
+```shell
+$ ./.cloud/test.sh [library][,library ...]
+```
+
+Examples:
+
+* `./.cloud/test.sh`
+* `./.cloud/test.sh java-accessapproval`
+* `./.cloud/test.sh java-asset,java-compute,java-container`
+
+### Workflow: Clean up GCP Project of Test Resources
+
+To clean-up (or “destroy”) all provisioned resources under Terraform’s management,
+invoke `cleanup.sh`:
+
+```shell
+$ ./.cloud/cleanup.sh
+```
+
+### Workflow: Add / Remove Client Libraries in an Existing Test Configuration
+
+Terraform will automatically manage the resources for a GCP project when regenerating the root
+module to include different client libraries. No additional commands are necessary to support this
+use-case.
+
+Example:
+
+```shell
+$ ./.cloud/setup.sh java-bigqueryconnection
+# Terraform provisions the project for java-bigqueryconnection's requirements.
+
+$ ./.cloud/setup.sh java-accessapproval
+# Terraform simultaneously destroys java-bigqueryconnection's resources that
+# are no longer being used, and provisions the project for java-accessapproval's
+# requirements.
+# Warning: This 'implicit removal' is prone to errors. If an error occurs, an
+# explicit invocation of 'cleanup.sh' may be required.
+
+$ ./.cloud/setup.sh java-accessapproval,java-texttospeech
+# Terraform maintains the java-accessapproval resources while also provisioning
+# the project for java-texttospeech's requirements.
+```

.cloud/cleanup.sh

@@ -0,0 +1,31 @@
+#
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+set -eo pipefail
+
+scriptDir="$(cd -P -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+pushd "$scriptDir" >/dev/null
+
+# Ensure GCP project environment variables are initialized.
+if [[ $(terraform state list) == "" ]]; then
+  echo "Nothing to destroy."
+  exit
+fi
+
+source ./helpers/gcloud-sync-env.sh
+source ./helpers/destroy.sh
+source ./helpers/gcloud-delete-project.sh
+
+popd >/dev/null

.cloud/helpers/apply.sh

@@ -0,0 +1,21 @@
+#
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+set -eo pipefail
+
+helperDir="$(cd -P -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+pushd "$helperDir/.." >/dev/null || exit
+terraform apply "generated.tfplan" || exit
+popd >/dev/null || exit

.cloud/helpers/common.sh

@@ -0,0 +1,71 @@
+#
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+set -eo pipefail
+
+# Find all directories starting with 'java-', sort them, then join
+# with ',' as the delimiter.
+function listAllModules() {
+  # Ensure current directory is repo root.
+  helperDir="$(cd -P -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+  pushd "$helperDir/../.." >/dev/null
+
+  ls -1 -d java-* | sort | paste -s -d, -
+
+  popd >/dev/null
+}
+
+# Replaces '-' with '_' to get a Terraform output-friendly label
+function getFriendlyOutputName() {
+  echo "$1" | tr '-' _
+}
+
+# Get the output object in JSON format for the given module.
+function getOutput() {
+  friendlyName=$(getFriendlyOutputName "$1")
+  terraform output -json "$friendlyName"
+}
+
+# Parse stdin and get the value associated with the given key.
+function parseJson() {
+  python3 -c "import sys, json; print(json.load(sys.stdin)['$1'])"
+}
+
+# Example use: getModuleOutput java-redis redis_network
+function getModuleOutput() {
+  getOutput "$1" | parseJson "$2"
+}
+
+# @returns exit code 0 if list $1 contains entry $2.
+function contains() {
+  echo "$1" | grep -w -q "$2"
+}
+
+# @returns a "new line"-delimited list of active terraform modules
+function getActiveTerraformModules() {
+  terraform state list | awk -F'[/.]' '{print $2}' | uniq
+}
+
+function getTerraformServiceAccountName() {
+  echo "terraform-service-account"
+}
+
+function getTerraformServiceAccountEmail() {
+  if [ -z "${GOOGLE_CLOUD_PROJECT}" ]; then
+    echo "GOOGLE_CLOUD_PROJECT must be defined."
+    exit 1
+  fi
+  echo "$(getTerraformServiceAccountName)@$GOOGLE_CLOUD_PROJECT.iam.gserviceaccount.com"
+}

.cloud/helpers/destroy.sh

@@ -0,0 +1,39 @@
+#
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+set -eo pipefail
+
+helperDir="$(cd -P -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+pushd "$helperDir/.." >/dev/null
+
+# Execute 'predestroy.sh' scripts for any active modules
+source ./helpers/common.sh
+allModules=$(listAllModules)
+activeModules=$(getActiveTerraformModules)
+IFS=','
+for module in $allModules; do
+  friendlyName=$(getFriendlyOutputName "$module")
+  if ! contains "$activeModules" "$friendlyName"; then
+    continue # Skip unless active.
+  fi
+
+  if [[ -f "../$module/.cloud/predestroy.sh" ]]; then
+    # shellcheck disable=SC1090
+    source "../$module/.cloud/predestroy.sh"
+  fi
+done
+
+terraform destroy -auto-approve
+popd >/dev/null