diff --git a/docs/technical-documentation/restoring_lost_data.md b/docs/technical-documentation/restoring_lost_data.md new file mode 100644 index 0000000000..03101e3950 --- /dev/null +++ b/docs/technical-documentation/restoring_lost_data.md @@ -0,0 +1,49 @@ +# Restoring Lost Data with Time-Travel + +If you've ever accidentally deleted important data either by using Terraform or by hand, there's still hope to recover the data. +By using the Snowflake's Time-Travel feature, you can restore lost data and undo those accidental deletions. + +> Note: Currently, the recovery process is predominantly manual, relying on SQL commands and the Terraform CLI. +We made a strategic decision not to integrate it as a provider feature at this time, as demand for this functionality was not significant. +Following the release of V1, we intend to reassess the topic of data recovery and UNDROP functionality to explore potential integration into the provider, evaluating its necessity and feasibility. + +You should be prepared beforehand by specifying how much of the historical data Snowflake should keep by setting the [DATA_RETENTION_TIME_IN_DAYS](https://docs.snowflake.com/en/sql-reference/parameters#data-retention-time-in-days) parameter. +When using our provider, you can set this by using one of our parameter-setting resources (like [snowflake_account_parameter](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs/resources/account_parameter) or [snowflake_object_parameter](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs/resources/object_parameter)) +or set it on the resource level (e.g. `data_retention_time_in_days` in [snowflake_database](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs/resources/database)). + +> Note: If some of the resources support `data_retention_time_in_days` parameter in Snowflake, but it's not available in the provider, we'll add it during [the resource preparation for V1](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/ROADMAP.md#preparing-essential-ga-objects-for-the-provider-v1). + +Now, with [DATA_RETENTION_TIME_IN_DAYS](https://docs.snowflake.com/en/sql-reference/parameters#data-retention-time-in-days) set up, +let's imagine we accidentally dropped a database that was managed by Terraform and contained a lot of important data we would like to recover. +But before we start, let's clearly understand the initial state of the database we'll be recovering. + +The configuration will contain only a database as we want to mainly focus on the data recovery from the Terraform point of view: +```terraform +resource "snowflake_database" "test" { + name = "TEST_DATABASE" +} +``` + +The rest of the objects will be created in SQL by hand (e.g. in the worksheet): +```sql +CREATE SCHEMA TEST_DATABASE.TEST_SCHEMA; +CREATE TABLE TEST_DATABASE.TEST_SCHEMA.TEST_TABLE(id INT, name STRING); +INSERT INTO TEST_DATABASE.TEST_SCHEMATEST_TABLE(id, name) VALUES (0, 'john'), (1, 'doe'); +``` +As you can see, we created a schema and a table with "important" data. + +As the next step we will remove the database from the configuration and run `terraform apply` that will trigger the `DROP DATABASE`. +Now, we're in a state where the data is lost and can only be recovered by Time-Travel. By checking up the [DATA_RETENTION_TIME_IN_DAYS](https://docs.snowflake.com/en/sql-reference/parameters#data-retention-time-in-days) +parameter, we can calculate if the dropped database can still be recovered or not (that's why it's important to prepare for such situations beforehand, +to avoid situations where the parameter was set to low value and the data is lost). + +To recover the database (and the data inside it), we have to call `UNDROP DATABASE TEST_DATABASE` manually. +To bring the database back to the Terraform configuration, we have to specify the same configuration as previously, but now, +instead of running `terraform apply` we have to import it by calling `terraform import 'TEST_DATABASE'`. +After successful import the `terraform plan` shouldn't produce any plan for the database. +To ensure all the important data we inserted before is there, we can call `SELECT * FROM TEST_DATABASE.TEST_SCHEMA.TEST_TABLE;`. + +To learn more about how to use Time-Travel, check out the links below: +1. [Understanding & using Time-Travel](https://docs.snowflake.com/en/user-guide/data-time-travel) +2. [DATA_RETENTION_TIME_IN_DAYS parameter](https://docs.snowflake.com/en/sql-reference/parameters#data-retention-time-in-days) +3. [UNDROP command with available objects to restore](https://docs.snowflake.com/en/sql-reference/sql/undrop) diff --git a/v1-preparations/CHANGES_BEFORE_V1.md b/v1-preparations/CHANGES_BEFORE_V1.md index 1bfab19df5..2f8f505102 100644 --- a/v1-preparations/CHANGES_BEFORE_V1.md +++ b/v1-preparations/CHANGES_BEFORE_V1.md @@ -1,6 +1,20 @@ # Changes before v1 -This document is a changelog of resources and datasources as part of the https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/ROADMAP.md#preparing-essential-ga-objects-for-the-provider-v1. Each provider version lists changes made in resources and datasources definitions during v1 preparations, like added, modified and removed fields. +This document is a changelog of resources and datasources as part of the https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/ROADMAP.md#preparing-essential-ga-objects-for-the-provider-v1. +Each provider version lists changes made in resources and datasources definitions during v1 preparations, like added, modified and removed fields. + +## Default values +For any resource that went through the rework as part of the [resource preparation for V1](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/ROADMAP.md#preparing-essential-ga-objects-for-the-provider-v1), +the behaviour for default values may change from the previous one. + +In the past, the provider copied defaults from Snowflake, creating a tight coupling between them. +However, this approach posed a challenge as the defaults on the Snowflake side could change and vary between accounts based on their configurations. + +Now, whenever the value is not specified in the configuration, we let the Snowflake fill out the default value for a given field +(if there is one). Using such defaults may lead to non-idempotent cases where the same configuration may +create a resource with slightly different configuration in Snowflake (depending on the Snowflake Edition and Version, +current account configuration, and most-likely other factors). That is why we recommend setting optional fields where +you want to ensure that the specified value has been set on the Snowflake side. ## v0.91.0 ➞ v0.92.0 ### snowflake_scim_integration resource changes