Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix unexpected crashes when using cloudflare_notification_policy with a filters attribute #1520

Closed

Conversation

0gajun
Copy link

@0gajun 0gajun commented Mar 18, 2022

Fixes #1189

Background

#1189

When running terraform plan for cloudflare_notification_policy with a filters attribute, I encountered the same issue with the following stack trace.

Stack trace from the terraform-provider-cloudflare_v3.8.0 plugin:

panic: Unknown validation type: 6

goroutine 90 [running]:
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.validateMapValues({0xd27c94, 0x7}, 0xc000390c30, 0xc00034db80, {0xc000390a60, 0x1, 0x1})
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/schema.go:1912 +0x969
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.schemaMap.validateMap(0xc2f5e0, {0xd27c94, 0x7}, {0xc2f5e0, 0xc000545290}, 0x8bdbac, 0xc0003d52e8, {0xc000390a60, 0x1, 0x1})
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/schema.go:1819 +0x271
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.schemaMap.validateType(0xc0003908e0, {0xd27c94, 0x7f496a}, {0xc2f5e0, 0xc000545290}, 0xc00034db80, 0xc1d4a0, {0xc000390a60, 0x1, 0x1})
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/schema.go:2111 +0x185
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.schemaMap.validate(0xc4edc0, {0xd27c94, 0x7}, 0xc00034db80, 0x0, {0xc000390a60, 0x1, 0x1})
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/schema.go:1534 +0x71a
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.schemaMap.validateObject(0xc000545680, {0x0, 0xc00047d2a8}, 0xcffce0, 0xc000390000, {0x13d7568, 0xc2f5e0, 0xc0005451d0})
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/schema.go:1970 +0x585
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.schemaMap.Validate(...)
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/schema.go:649
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).Validate(0xc0003397a0, 0xc000318480)
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:542 +0x4d
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Provider).ValidateResource(0xe56d80, {0xc0003a74c0, 0x1e}, 0xc0005451d0)
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/provider.go:254 +0xf2
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ValidateResourceTypeConfig(0xc00000c138, {0xc0003fef40, 0xd25dbf}, 0xc00047d1d0)
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:230 +0x126
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ValidateResourceTypeConfig(0xc00007e880, {0xe56680, 0xc00050b260}, 0xc0003fed80)
github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:503 +0x2e2
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ValidateResourceTypeConfig_Handler({0xcf6a20, 0xc00007e880}, {0xe56680, 0xc00050b260}, 0xc0000a9980, 0x0)
github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:272 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0002988c0, {0xe64ea8, 0xc0001a6480}, 0xc000561d40, 0xc000399530, 0x13982b0, 0x0)
google.golang.org/[email protected]/server.go:1297 +0xccf
google.golang.org/grpc.(*Server).handleStream(0xc0002988c0, {0xe64ea8, 0xc0001a6480}, 0xc000561d40, 0x0)
google.golang.org/[email protected]/server.go:1626 +0xa2a
google.golang.org/grpc.(*Server).serveStreams.func1.2()
google.golang.org/[email protected]/server.go:941 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
google.golang.org/[email protected]/server.go:939 +0x294

Currently, TypeMap doesn't support non-primitive type values.
ref : hashicorp/terraform-plugin-sdk#62

When using non-primitive values with TypeMap, terraform crashes at runtime with a following error message.

panic: Unknown validation type: 6

This error is originated at
https://github.com/hashicorp/terraform-plugin-sdk/blob/v2.11.0/helper/schema/schema.go#L2017
Only TypeBool, TypeInt, TypeFloat and TypeString are supported.

This is why terraform crashes.

So, in this change, use TypeList with new custom resource to store non-primitive values.

NOTE

I also couldn't find any documentation about a filter parameters. ( #1406 )

Therefore, currently this change doesn't support all possible filters. (now supports zones and services filters only)

Could you help me to support all possible filters, Cloudflare staffs?

@github-actions
Copy link
Contributor

Oops! It looks like no changelog entry is attached to this PR. Please include a release note as described in https://github.com/cloudflare/terraform-provider-cloudflare/blob/master/docs/changelog-process.md.

Example:

```release-note:TYPE
Release note
```

If you do not require a release note to be included, please add the workflow/skip-changelog-entry label.

@0gajun 0gajun force-pushed the dont_use_map_for_non_primitive_values branch from f4ca45d to 9bf1bf9 Compare March 18, 2022 14:40
@jacobbednarz
Copy link
Member

can you please add a test covering the functionality change here? we'll also need to update the website documentation for this resource.

Currently, TypeMap doesn't support non-primitive type values.
ref : hashicorp/terraform-plugin-sdk#62

When using non-primitive values with TypeMap, terraform crashes at
runtime with a following error message.

```
panic: Unknown validation type: 6
```

This error is originated at
https://github.com/hashicorp/terraform-plugin-sdk/blob/v2.11.0/helper/schema/schema.go#L2017
Only TypeBool, TypeInt, TypeFloat and TypeString are supported.

So, in this change, use TypeList with new custom resource to store
non-primitive values.
@0gajun 0gajun force-pushed the dont_use_map_for_non_primitive_values branch from 9bf1bf9 to 7e1f0cd Compare March 21, 2022 07:31
@0gajun
Copy link
Author

0gajun commented Mar 21, 2022

Thanks for your comment 😄

I added a new acceptance test using a filters attribute.

However, I have not run the acceptance test in my personal environment yet because the clickhouse_alert_fw_ent_anomaly alert requires Enterprise plan of Cloudflare.
(Maybe I cannot make an Enterprise contract only for this PR...)

(I tested my code with development overrides for provider developers in my real dev environment, but I think acceptance tests should be run in an isolated environment only for the tests)

Do you have any good idea to run the acceptance test by myself?

@jacobbednarz
Copy link
Member

i have a dedicated account for running the acceptance tests in. https://github.com/cloudflare/terraform-provider-cloudflare/blob/master/GNUmakefile#L23 is the target (with a few other environment variables that i use)

looks like these changes are breaking the existing test suite

TF_ACC=1 go test $(go list ./...) -v -run "^TestAccCloudflareNotificationPolicy_" -count 1 -parallel 1 -timeout 120m -parallel 1
?   	github.com/cloudflare/terraform-provider-cloudflare	[no test files]
=== RUN   TestAccCloudflareNotificationPolicy_Basic
panic: filters: '': source data must be an array or slice, got map

goroutine 223 [running]:
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*ResourceData).Set(0xc000697680, {0x1ae3bcc, 0x7}, {0x19c8ea0, 0xc00069e600})
	/Users/jacob/.asdf/installs/golang/1.17.7/packages/pkg/mod/github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource_data.go:230 +0x2a5
github.com/cloudflare/terraform-provider-cloudflare/cloudflare.resourceCloudflareNotificationPolicyRead(0xc000697680, {0x1add940, 0xc00016a3c0})
	/Users/jacob/go/src/github.com/cloudflare/terraform-provider-cloudflare/cloudflare/resource_cloudflare_notification_policy.go:58 +0x3ed
github.com/cloudflare/terraform-provider-cloudflare/cloudflare.resourceCloudflareNotificationPolicyCreate(0x0, {0x1add940, 0xc00016a3c0})
	/Users/jacob/go/src/github.com/cloudflare/terraform-provider-cloudflare/cloudflare/resource_cloudflare_notification_policy.go:39 +0x2c5
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).create(0x1c6e140, {0x1c6e140, 0xc00028d590}, 0xd, {0x1add940, 0xc00016a3c0})
	/Users/jacob/.asdf/installs/golang/1.17.7/packages/pkg/mod/github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:330 +0x178
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).Apply(0xc0003de000, {0x1c6e140, 0xc00028d590}, 0xc0003fc410, 0xc000697500, {0x1add940, 0xc00016a3c0})
	/Users/jacob/.asdf/installs/golang/1.17.7/packages/pkg/mod/github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:472 +0x9ba
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ApplyResourceChange(0xc000488270, {0x1c6e098, 0xc0006eedc0}, 0xc0006eb090)
	/Users/jacob/.asdf/installs/golang/1.17.7/packages/pkg/mod/github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:1021 +0xdaa
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ApplyResourceChange(0xc0002ce000, {0x1c6e140, 0xc00028cdb0}, 0xc0002d67e0)
	/Users/jacob/.asdf/installs/golang/1.17.7/packages/pkg/mod/github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:812 +0x56b
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler({0x1aad420, 0xc0002ce000}, {0x1c6e140, 0xc00028cdb0}, 0xc00068f1a0, 0x0)
	/Users/jacob/.asdf/installs/golang/1.17.7/packages/pkg/mod/github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:385 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000316380, {0x1c80410, 0xc00050bba0}, 0xc0004779e0, 0xc00071cd50, 0x22bf180, 0x0)
	/Users/jacob/.asdf/installs/golang/1.17.7/packages/pkg/mod/google.golang.org/[email protected]/server.go:1282 +0xccf
google.golang.org/grpc.(*Server).handleStream(0xc000316380, {0x1c80410, 0xc00050bba0}, 0xc0004779e0, 0x0)
	/Users/jacob/.asdf/installs/golang/1.17.7/packages/pkg/mod/google.golang.org/[email protected]/server.go:1619 +0xa2a
google.golang.org/grpc.(*Server).serveStreams.func1.2()
	/Users/jacob/.asdf/installs/golang/1.17.7/packages/pkg/mod/google.golang.org/[email protected]/server.go:921 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	/Users/jacob/.asdf/installs/golang/1.17.7/packages/pkg/mod/google.golang.org/[email protected]/server.go:919 +0x294
FAIL	github.com/cloudflare/terraform-provider-cloudflare/cloudflare	6.969s
?   	github.com/cloudflare/terraform-provider-cloudflare/tools/cmd/changelog-check	[no test files]
?   	github.com/cloudflare/terraform-provider-cloudflare/tools/cmd/maintainer-only-file-check	[no test files]
?   	github.com/cloudflare/terraform-provider-cloudflare/tools/cmd/tf-log-check	[no test files]
?   	github.com/cloudflare/terraform-provider-cloudflare/version	[no test files]
FAIL
make: *** [testacc] Error 1

@@ -155,7 +160,7 @@ func buildNotificationPolicy(d *schema.ResourceData) cloudflare.NotificationPoli
}

if filters, ok := d.GetOk("filters"); ok {
notificationPolicy.Filters = filters.(map[string][]string)
notificationPolicy.Filters = filters.([]interface{})[0].(map[string][]string)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if i'm understanding this correctly, you'll need to build this map[string][]string instead of casting it. looping over filters and appending should do it.

Cause filters and its child attributes don't care about ordering of
values.
@0gajun
Copy link
Author

0gajun commented Mar 22, 2022

Thanks for your kind review and running acceptance tests on your environment !

I changed filter's type from TypeList to TypeSet.

fe50c84

Maybe this change fixes the broken existing test suite. Could you run the acceptance test again when you have time?

@@ -25,11 +25,26 @@ func resourceCloudflareNotificationPolicySchema() map[string]*schema.Schema {
Required: true,
},
"filters": {
Type: schema.TypeMap,
Type: schema.TypeSet,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you still want this as a TypeList otherwise it won't be addressable as filters.0...

Suggested change
Type: schema.TypeSet,
Type: schema.TypeList,

resource.TestCheckResourceAttr(resourceName, "alert_type", "clickhouse_alert_fw_ent_anomaly"),
resource.TestCheckResourceAttr(resourceName, "account_id", accountID),
resource.TestCheckResourceAttr(resourceName, "filters.0.services.#", "1"),
resource.TestCheckResourceAttr(resourceName, "filters.0.services.0", "waf"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

services.0 isn't directly addressable as the index is a hash of the items; you'll need to use TestCheckTypeSetElemAttr instead.

@jacobbednarz
Copy link
Member

superseded by #1542

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Crashes on cloudflare_notification_policy
2 participants