Skip to content

Commit

Permalink
feat: Introduce security integrations datasource (#2892)
Browse files Browse the repository at this point in the history
Added security integrations datasource, for all types of security
integrations:
- new datasource implemented
- new datasource initially tested
- examples added
- docs generated

Additional changes:
- slightly improved show output schemas generator to generate schema for
any other SDK object (with some todos for later)
- fixed previous datasources descriptions
- fixed migration guide entry

What is missing:
- describe schema is not created (waiting for all security integrations
PRs)
- testing all integrations (waiting for all security integrations PRs)
  • Loading branch information
sfc-gh-asawicki committed Jun 28, 2024
1 parent e11e608 commit 7f6c657
Show file tree
Hide file tree
Showing 24 changed files with 609 additions and 38 deletions.
12 changes: 11 additions & 1 deletion MIGRATION_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,16 @@ across different versions.
### old grant resources removal
Following the [announcement](https://github.com/Snowflake-Labs/terraform-provider-snowflake/discussions/2736) we have removed the old grant resources. The two resources [snowflake_role_ownership_grant](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs/resources/role_ownership_grant) and [snowflake_user_ownership_grant](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs/resources/user_ownership_grant) were not listed in the announcement, but they were also marked as deprecated ones. We are removing them too to conclude the grants redesign saga.

### *(new feature)* snowflake_security_integrations datasource
Added a new datasource enabling querying and filtering all types of security integrations. Notes:
- all results are stored in `security_integrations` field.
- `like` field enables security integrations filtering.
- SHOW SECURITY INTEGRATIONS output is enclosed in `show_output` field inside `security_integrations`.
- Output from **DESC SECURITY INTEGRATION** (which can be turned off by declaring `with_describe = false`, **it's turned on by default**) is enclosed in `describe_output` field inside `security_integrations`.
**DESC SECURITY INTEGRATION** returns different properties based on the integration type. Consult the documentation to check which ones will be filled for which integration.
The additional parameters call **DESC SECURITY INTEGRATION** (with `with_describe` turned on) **per security integration** returned by **SHOW SECURITY INTEGRATIONS**.
It's important to limit the records and calls to Snowflake to the minimum. That's why we recommend assessing which information you need from the data source and then providing strong filters and turning off additional fields for better plan performance.

### snowflake_scim_integration resource changes
#### *(behavior change)* Changed behavior of `sync_password`

Expand Down Expand Up @@ -82,7 +92,7 @@ To easily handle three-value logic (true, false, unknown) in provider's configs,
#### *(note)* `resource_monitor` validation and diff suppression
`resource_monitor` is an identifier and handling logic may be still slightly changed as part of https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/ROADMAP.md#identifiers-rework. It should be handled automatically (without needed manual actions on user side), though, but it is not guaranteed.

#### *(behavior change)* snowflake_databases datasource
#### *(behavior change)* snowflake_warehouses datasource
- Added `like` field to enable warehouse filtering
- Added missing fields returned by SHOW WAREHOUSES and enclosed its output in `show_output` field.
- Added outputs from **DESC WAREHOUSE** and **SHOW PARAMETERS IN WAREHOUSE** (they can be turned off by declaring `with_describe = false` and `with_parameters = false`, **they're turned on by default**).
Expand Down
4 changes: 2 additions & 2 deletions docs/data-sources/databases.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
page_title: "snowflake_databases Data Source - terraform-provider-snowflake"
subcategory: ""
description: |-
Datasource used to get details of filtered databases. Filtering is aligned with the current possibilities for SHOW DATABASES (https://docs.snowflake.com/en/sql-reference/sql/show-databases) query (`like`, 'starts_with', and `limit` are all supported. The results of SHOW, DESCRIBE, and SHOW PARAMETERS IN are encapsulated in one output collection.
Datasource used to get details of filtered databases. Filtering is aligned with the current possibilities for SHOW DATABASES https://docs.snowflake.com/en/sql-reference/sql/show-databases query (like, starts_with, and limit are all supported). The results of SHOW, DESCRIBE, and SHOW PARAMETERS IN are encapsulated in one output collection.
---

# snowflake_databases (Data Source)

Datasource used to get details of filtered databases. Filtering is aligned with the current possibilities for [SHOW DATABASES]((https://docs.snowflake.com/en/sql-reference/sql/show-databases) query (`like`, 'starts_with', and `limit` are all supported). The results of SHOW, DESCRIBE, and SHOW PARAMETERS IN are encapsulated in one output collection.
Datasource used to get details of filtered databases. Filtering is aligned with the current possibilities for [SHOW DATABASES](https://docs.snowflake.com/en/sql-reference/sql/show-databases) query (`like`, `starts_with`, and `limit` are all supported). The results of SHOW, DESCRIBE, and SHOW PARAMETERS IN are encapsulated in one output collection.

## Example Usage

Expand Down
125 changes: 125 additions & 0 deletions docs/data-sources/security_integrations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
---
page_title: "snowflake_security_integrations Data Source - terraform-provider-snowflake"
subcategory: ""
description: |-
Datasource used to get details of filtered security integrations. Filtering is aligned with the current possibilities for SHOW SECURITY INTEGRATIONS https://docs.snowflake.com/en/sql-reference/sql/show-integrations query (only like is supported). The results of SHOW and DESCRIBE are encapsulated in one output collection security_integrations.
---

# snowflake_security_integrations (Data Source)

Datasource used to get details of filtered security integrations. Filtering is aligned with the current possibilities for [SHOW SECURITY INTEGRATIONS](https://docs.snowflake.com/en/sql-reference/sql/show-integrations) query (only `like` is supported). The results of SHOW and DESCRIBE are encapsulated in one output collection `security_integrations`.

## Example Usage

```terraform
# Simple usage
data "snowflake_security_integrations" "simple" {
}
output "simple_output" {
value = data.snowflake_security_integrations.simple.security_integrations
}
# Filtering (like)
data "snowflake_security_integrations" "like" {
like = "security-integration-name"
}
output "like_output" {
value = data.snowflake_security_integrations.like.security_integrations
}
# Filtering by prefix (like)
data "snowflake_security_integrations" "like_prefix" {
like = "prefix%"
}
output "like_prefix_output" {
value = data.snowflake_security_integrations.like_prefix.security_integrations
}
# Without additional data (to limit the number of calls make for every found security integration)
data "snowflake_security_integrations" "only_show" {
# with_describe is turned on by default and it calls DESCRIBE SECURITY INTEGRATION for every security integration found and attaches its output to security_integrations.*.describe_output field
with_describe = false
}
output "only_show_output" {
value = data.snowflake_security_integrations.only_show.security_integrations
}
# Ensure the number of security_integrations is equal to at least one element (with the use of postcondition)
data "snowflake_security_integrations" "assert_with_postcondition" {
like = "security-integration-name%"
lifecycle {
postcondition {
condition = length(self.security_integrations) > 0
error_message = "there should be at least one security integration"
}
}
}
# Ensure the number of security_integrations is equal to at exactly one element (with the use of check block)
check "security_integration_check" {
data "snowflake_security_integrations" "assert_with_check_block" {
like = "security-integration-name"
}
assert {
condition = length(data.snowflake_security_integrations.assert_with_check_block.security_integrations) == 1
error_message = "security integrations filtered by '${data.snowflake_security_integrations.assert_with_check_block.like}' returned ${length(data.snowflake_security_integrations.assert_with_check_block.security_integrations)} security integrations where one was expected"
}
}
```

<!-- schema generated by tfplugindocs -->
## Schema

### Optional

- `like` (String) Filters the output with **case-insensitive** pattern, with support for SQL wildcard characters (`%` and `_`).
- `with_describe` (Boolean) Runs DESC SECURITY INTEGRATION for each security integration returned by SHOW SECURITY INTEGRATIONS. The output of describe is saved to the description field. By default this value is set to true.

### Read-Only

- `id` (String) The ID of this resource.
- `security_integrations` (List of Object) Holds the aggregated output of all security integrations details queries. (see [below for nested schema](#nestedatt--security_integrations))

<a id="nestedatt--security_integrations"></a>
### Nested Schema for `security_integrations`

Read-Only:

- `describe_output` (List of Object) (see [below for nested schema](#nestedobjatt--security_integrations--describe_output))
- `show_output` (List of Object) (see [below for nested schema](#nestedobjatt--security_integrations--show_output))

<a id="nestedobjatt--security_integrations--describe_output"></a>
### Nested Schema for `security_integrations.describe_output`

Read-Only:

- `todo` (List of Object) (see [below for nested schema](#nestedobjatt--security_integrations--describe_output--todo))

<a id="nestedobjatt--security_integrations--describe_output--todo"></a>
### Nested Schema for `security_integrations.describe_output.todo`

Read-Only:

- `default` (String)
- `name` (String)
- `type` (String)
- `value` (String)



<a id="nestedobjatt--security_integrations--show_output"></a>
### Nested Schema for `security_integrations.show_output`

Read-Only:

- `category` (String)
- `comment` (String)
- `created_on` (String)
- `enabled` (Boolean)
- `integration_type` (String)
- `name` (String)
4 changes: 2 additions & 2 deletions docs/data-sources/warehouses.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
page_title: "snowflake_warehouses Data Source - terraform-provider-snowflake"
subcategory: ""
description: |-
Datasource used to get details of filtered warehouses. Filtering is aligned with the current possibilities for [SHOW WAREHOUSES]((https://docs.snowflake.com/en/sql-reference/sql/show-warehouses) query (only like is supported). The results of SHOW, DESCRIBE, and SHOW PARAMETERS IN are encapsulated in one output collection.
Datasource used to get details of filtered warehouses. Filtering is aligned with the current possibilities for SHOW WAREHOUSES https://docs.snowflake.com/en/sql-reference/sql/show-warehouses query (only like is supported). The results of SHOW, DESCRIBE, and SHOW PARAMETERS IN are encapsulated in one output collection.
---

# snowflake_warehouses (Data Source)

Datasource used to get details of filtered warehouses. Filtering is aligned with the current possibilities for [SHOW WAREHOUSES]((https://docs.snowflake.com/en/sql-reference/sql/show-warehouses) query (only `like` is supported). The results of SHOW, DESCRIBE, and SHOW PARAMETERS IN are encapsulated in one output collection.
Datasource used to get details of filtered warehouses. Filtering is aligned with the current possibilities for [SHOW WAREHOUSES](https://docs.snowflake.com/en/sql-reference/sql/show-warehouses) query (only `like` is supported). The results of SHOW, DESCRIBE, and SHOW PARAMETERS IN are encapsulated in one output collection.

## Example Usage

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Simple usage
data "snowflake_security_integrations" "simple" {
}

output "simple_output" {
value = data.snowflake_security_integrations.simple.security_integrations
}

# Filtering (like)
data "snowflake_security_integrations" "like" {
like = "security-integration-name"
}

output "like_output" {
value = data.snowflake_security_integrations.like.security_integrations
}

# Filtering by prefix (like)
data "snowflake_security_integrations" "like_prefix" {
like = "prefix%"
}

output "like_prefix_output" {
value = data.snowflake_security_integrations.like_prefix.security_integrations
}

# Without additional data (to limit the number of calls make for every found security integration)
data "snowflake_security_integrations" "only_show" {
# with_describe is turned on by default and it calls DESCRIBE SECURITY INTEGRATION for every security integration found and attaches its output to security_integrations.*.describe_output field
with_describe = false
}

output "only_show_output" {
value = data.snowflake_security_integrations.only_show.security_integrations
}

# Ensure the number of security_integrations is equal to at least one element (with the use of postcondition)
data "snowflake_security_integrations" "assert_with_postcondition" {
like = "security-integration-name%"
lifecycle {
postcondition {
condition = length(self.security_integrations) > 0
error_message = "there should be at least one security integration"
}
}
}

# Ensure the number of security_integrations is equal to at exactly one element (with the use of check block)
check "security_integration_check" {
data "snowflake_security_integrations" "assert_with_check_block" {
like = "security-integration-name"
}

assert {
condition = length(data.snowflake_security_integrations.assert_with_check_block.security_integrations) == 1
error_message = "security integrations filtered by '${data.snowflake_security_integrations.assert_with_check_block.like}' returned ${length(data.snowflake_security_integrations.assert_with_check_block.security_integrations)} security integrations where one was expected"
}
}
3 changes: 1 addition & 2 deletions pkg/datasources/databases.go
Original file line number Diff line number Diff line change
Expand Up @@ -90,12 +90,11 @@ var databasesSchema = map[string]*schema.Schema{
},
}

// Databases the Snowflake current account resource.
func Databases() *schema.Resource {
return &schema.Resource{
ReadContext: ReadDatabases,
Schema: databasesSchema,
Description: "Datasource used to get details of filtered databases. Filtering is aligned with the current possibilities for [SHOW DATABASES]((https://docs.snowflake.com/en/sql-reference/sql/show-databases) query (`like`, 'starts_with', and `limit` are all supported). The results of SHOW, DESCRIBE, and SHOW PARAMETERS IN are encapsulated in one output collection.",
Description: "Datasource used to get details of filtered databases. Filtering is aligned with the current possibilities for [SHOW DATABASES](https://docs.snowflake.com/en/sql-reference/sql/show-databases) query (`like`, `starts_with`, and `limit` are all supported). The results of SHOW, DESCRIBE, and SHOW PARAMETERS IN are encapsulated in one output collection.",
}
}

Expand Down
102 changes: 102 additions & 0 deletions pkg/datasources/security_integrations.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
package datasources

import (
"context"

"github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider"
"github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/schemas"
"github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk"
"github.com/hashicorp/terraform-plugin-sdk/v2/diag"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
)

var securityIntegrationsSchema = map[string]*schema.Schema{
"with_describe": {
Type: schema.TypeBool,
Optional: true,
Default: true,
Description: "Runs DESC SECURITY INTEGRATION for each security integration returned by SHOW SECURITY INTEGRATIONS. The output of describe is saved to the description field. By default this value is set to true.",
},
"like": {
Type: schema.TypeString,
Optional: true,
Description: "Filters the output with **case-insensitive** pattern, with support for SQL wildcard characters (`%` and `_`).",
},
"security_integrations": {
Type: schema.TypeList,
Computed: true,
Description: "Holds the aggregated output of all security integrations details queries.",
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"show_output": {
Type: schema.TypeList,
Computed: true,
Description: "Holds the output of SHOW SECURITY INTEGRATIONS.",
Elem: &schema.Resource{
Schema: schemas.ShowSecurityIntegrationSchema,
},
},
"describe_output": {
Type: schema.TypeList,
Computed: true,
Description: "Holds the output of DESCRIBE SECURITY INTEGRATIONS.",
Elem: &schema.Resource{
Schema: schemas.SecurityIntegrationDescribeSchema,
},
},
},
},
},
}

func SecurityIntegrations() *schema.Resource {
return &schema.Resource{
ReadContext: ReadSecurityIntegrations,
Schema: securityIntegrationsSchema,
Description: "Datasource used to get details of filtered security integrations. Filtering is aligned with the current possibilities for [SHOW SECURITY INTEGRATIONS](https://docs.snowflake.com/en/sql-reference/sql/show-integrations) query (only `like` is supported). The results of SHOW and DESCRIBE are encapsulated in one output collection `security_integrations`.",
}
}

func ReadSecurityIntegrations(ctx context.Context, d *schema.ResourceData, meta any) diag.Diagnostics {
client := meta.(*provider.Context).Client
showRequest := sdk.NewShowSecurityIntegrationRequest()

if likePattern, ok := d.GetOk("like"); ok {
showRequest.WithLike(sdk.Like{
Pattern: sdk.String(likePattern.(string)),
})
}

securityIntegrations, err := client.SecurityIntegrations.Show(ctx, showRequest)
if err != nil {
return diag.FromErr(err)
}
d.SetId("security_integrations_read")

flattenedSecurityIntegrations := make([]map[string]any, len(securityIntegrations))

for i, securityIntegration := range securityIntegrations {
securityIntegration := securityIntegration
var securityIntegrationDescriptions []map[string]any
if d.Get("with_describe").(bool) {
descriptions, err := client.SecurityIntegrations.Describe(ctx, securityIntegration.ID())
if err != nil {
return diag.FromErr(err)
}
securityIntegrationDescriptions = make([]map[string]any, 1)
securityIntegrationDescriptions[0] = schemas.SecurityIntegrationsDescriptionsToSchema(descriptions)
}

flattenedSecurityIntegrations[i] = map[string]any{
"show_output": []map[string]any{schemas.SecurityIntegrationToSchema(&securityIntegration)},
"describe_output": securityIntegrationDescriptions,
}
}

err = d.Set("security_integrations", flattenedSecurityIntegrations)
if err != nil {
return diag.FromErr(err)
}

return nil
}
Loading

0 comments on commit 7f6c657

Please sign in to comment.