Skip to content
This repository has been archived by the owner on May 15, 2023. It is now read-only.

Conversation

modular-magician
Copy link
Collaborator

Description

Fixes hashicorp/terraform-provider-google#8994

This PR lets Terraform users set the retention period of historical versions of a Spanner database's data and schema for point in time recovery (PITR).

The retention period of the data used for PITR can be changed by running a DDL statement to update the database's options:

-- Google Standard SQL
ALTER DATABASE `example-db` SET OPTIONS (version_retention_period="7d";

-- Postgres
ALTER DATABASE `example-db` SET spanner.version_retention_period TO "7d";

See usage notes section here & docs about equivalent DDL for Postgres

These DDL statements can be sent through the REST API and executed at either:

  1. Creation of a new database via the create endpoint
    • as an entry in the extraStatements array in the request body
    • ⚠️ not allowed for Postgres ⚠️
      • If you provide any extraStatements in the create request for a Postgres database, you get this error: DDL statements other than <CREATE DATABASE> are not allowed in database creation request for PostgreSQL-enabled databases
  2. Updating a database via the updateDdl endpoint
    • OK for both database dialects

Approach

I've used the value of the version_retention_period field to create an ALTER DDL statement like above and add it to the array of statements in the request bodies of requests sent to the create or updateDdl endpoints

Edit 2022-06-27 : I refactored the code for creating databases so that DDL statements (both originating from ddl and version_retention_period fields) are executed in the database via a separate API call. After the database is made using the create endpoint the provider runs DDL against the new database using the updateDdl endpoint, all within the creation process in Terraform.

Postgres issues

The DDL restriction when making Postgres databases has been met by the initial version of this PR by skipping setting the retention time during creation, even if the version_retention_period is set in the Terraform configuration. When a plan is generated a 2nd time from the Terraform configuration it will plan to update the retention time on the Postgres DB.

This might be confusing to practitioners, but seems like the need to re-apply plans for Postgres databases isn't new and is mentioned in the docs.

Edit 2022-06-27 : By separating all execution of DDL into an API call separate to the initial create API call this means Postgres's limitations in Spanner don't require 2nd apply steps

Open questions

  • How does this approach work with acceptance tests, which try to prevent scenarios where 2nd plans from the same config contain changes still. Is that a sign that this should be approached differently?
  • Would a more explicit error that disallows version_retention_period being set before making the Postgres DB be better? That would avoid the confusion of needing to apply a plan twice to reach the desired goal
  • Should there be any protection against users setting the retention time via an ALTER statement in the ddl parameter of the Terraform provider? The only issue would be if someone added version_retention_period to their config but then resumed setting it via ddl (version_retention_period would always re-assert itself on the next plan&apply)

PR steps

If this PR is for Terraform, I acknowledge that I have:

  • Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
  • Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
  • Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
  • Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
  • Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

spanner: Added field `version_retention_period` to `google_spanner_database` resource
spanner: Fixed issue where `ddl` and `version_retention_period` could not be set when first creating `google_spanner_database` using POSTGRESQL

Derived from GoogleCloudPlatform/magic-modules#6141

@modular-magician modular-magician requested a review from a team as a code owner June 29, 2022 16:55
@modular-magician modular-magician requested review from iyabchen and removed request for a team June 29, 2022 16:55
@modular-magician modular-magician merged commit 3c5ce58 into GoogleCloudPlatform:main Jun 29, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add support version_retention_period with google_spanner_database
1 participant