Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add version retention period to Spanner database resource #11982

Conversation

modular-magician
Copy link
Collaborator

Description

Fixes #8994

This PR lets Terraform users set the retention period of historical versions of a Spanner database's data and schema for point in time recovery (PITR).

The retention period of the data used for PITR can be changed by running a DDL statement to update the database's options:

-- Google Standard SQL
ALTER DATABASE `example-db` SET OPTIONS (version_retention_period="7d";

-- Postgres
ALTER DATABASE `example-db` SET spanner.version_retention_period TO "7d";

See usage notes section here & docs about equivalent DDL for Postgres

These DDL statements can be sent through the REST API and executed at either:

  1. Creation of a new database via the create endpoint
    • as an entry in the extraStatements array in the request body
    • ⚠️ not allowed for Postgres ⚠️
      • If you provide any extraStatements in the create request for a Postgres database, you get this error: DDL statements other than <CREATE DATABASE> are not allowed in database creation request for PostgreSQL-enabled databases
  2. Updating a database via the updateDdl endpoint
    • OK for both database dialects

Approach

I've used the value of the version_retention_period field to create an ALTER DDL statement like above and add it to the array of statements in the request bodies of requests sent to the create or updateDdl endpoints

Edit 2022-06-27 : I refactored the code for creating databases so that DDL statements (both originating from ddl and version_retention_period fields) are executed in the database via a separate API call. After the database is made using the create endpoint the provider runs DDL against the new database using the updateDdl endpoint, all within the creation process in Terraform.

Postgres issues

The DDL restriction when making Postgres databases has been met by the initial version of this PR by skipping setting the retention time during creation, even if the version_retention_period is set in the Terraform configuration. When a plan is generated a 2nd time from the Terraform configuration it will plan to update the retention time on the Postgres DB.

This might be confusing to practitioners, but seems like the need to re-apply plans for Postgres databases isn't new and is mentioned in the docs.

Edit 2022-06-27 : By separating all execution of DDL into an API call separate to the initial create API call this means Postgres's limitations in Spanner don't require 2nd apply steps

Open questions

  • How does this approach work with acceptance tests, which try to prevent scenarios where 2nd plans from the same config contain changes still. Is that a sign that this should be approached differently?
  • Would a more explicit error that disallows version_retention_period being set before making the Postgres DB be better? That would avoid the confusion of needing to apply a plan twice to reach the desired goal
  • Should there be any protection against users setting the retention time via an ALTER statement in the ddl parameter of the Terraform provider? The only issue would be if someone added version_retention_period to their config but then resumed setting it via ddl (version_retention_period would always re-assert itself on the next plan&apply)

PR steps

If this PR is for Terraform, I acknowledge that I have:

  • Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
  • Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
  • Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
  • Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
  • Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

spanner: Added field `version_retention_period` to `google_spanner_database` resource
spanner: Fixed issue where `ddl` and `version_retention_period` could not be set when first creating `google_spanner_database` using POSTGRESQL

Derived from GoogleCloudPlatform/magic-modules#6141

@modular-magician modular-magician merged commit 792521c into hashicorp:main Jun 29, 2022
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 30, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add support version_retention_period with google_spanner_database
1 participant